# New in llama.cpp: Model Management

DevFeed: [New in llama.cpp: Model Management](<https://devfeed.tech/articles/new-in-llama-cpp-model-management-7216.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ggml-org/model-management-in-llamacpp>)

Author: Xuan-Son Nguyen; Victor Mustar

Published: 2025-12-11T15:47:44Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI, ML & Data Engineering](<https://devfeed.tech/topics/ai-ml-data-engineering.md>), [Processes](<https://devfeed.tech/topics/processes.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [llms](<https://devfeed.tech/tags/llms.md>), [management](<https://devfeed.tech/tags/management.md>), [models](<https://devfeed.tech/tags/models.md>), [process](<https://devfeed.tech/tags/process.md>), [routing](<https://devfeed.tech/tags/routing.md>), [server](<https://devfeed.tech/tags/server.md>)

## AI overview

llama.cpp introduces model management for its local, OpenAI-compatible LLM server. Router mode discovers GGUF models, loads them on demand, evicts least-recently-used models, and routes requests by model field.

## Source excerpt

Reminder: llama.cpp server is a lightweight, OpenAI-compatible HTTP server for running LLMs locally. This feature was a popular request to bring Ollama-style model management to llama.cpp. It uses a multi-process architecture where each model runs in its own process, so if one model crashes, others remain unaffected. Start the server in router mode by not specifying a model: This auto-discovers models from your llama.cpp cache ( or ).