feat: add per-model GPU and RAM placement
This commit is contained in:
@@ -47,7 +47,15 @@ The Available downloads controls support:
|
||||
|
||||
Popularity and date ordering use upstream metadata only; the plugin does not invent popularity, dates, RAM requirements, or token metrics.
|
||||
|
||||
## Ollama capacity warnings
|
||||
## Per-model placement
|
||||
|
||||
Each installed model in the Model pool has a persistent placement selector:
|
||||
|
||||
- **GPU + RAM (automatic offload)**: Ollama uses GPU layers where it can and keeps the remainder in system RAM. This is the default.
|
||||
- **RAM only (CPU)**: the plugin sends Ollama `num_gpu: 0`, preventing GPU layer offload for that model.
|
||||
|
||||
Placement is sent both when loading models and when chatting, so a RAM-only model is not silently reloaded with GPU offload. RAM-only models will not increase GPU VRAM usage; GPU+RAM models can still be evicted by Ollama if the GPU/device-memory scheduler cannot fit the runner.
|
||||
|
||||
|
||||
When Ollama accepts a load request but evicts one model while starting another, the dashboard displays an amber capacity warning rather than a red plugin error. On the verified host, Ollama logged that a 27.9 GiB runner would exceed available device memory with approximately 2.5 GiB GPU memory free. The T600 has 4 GiB VRAM, so two large multimodal models cannot be guaranteed resident simultaneously by the plugin. The actual resident set remains authoritative through `/api/ps`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user