feat: add Ollama chat and live memory telemetry
This commit is contained in:
@@ -1,28 +1,34 @@
|
||||
# Hermes Ollama Models
|
||||
|
||||
Native-like Hermes dashboard plugin for inspecting and managing a local Ollama installation.
|
||||
Native-like Hermes dashboard plugin for local Ollama model management and chat.
|
||||
|
||||
## Included
|
||||
## Chat capabilities
|
||||
|
||||
- Installed and currently loaded Ollama model inventory
|
||||
- Model size, loaded memory, estimated RAM, quantization, context, capabilities, and strengths
|
||||
- Dense versus MoE classification
|
||||
- Search, popular models, family variants, downloads, updates, and removal actions
|
||||
- MLX model exclusion
|
||||
- RAM-aware Popular view for the current 30 GiB host
|
||||
- Select an installed Ollama model and load it into memory
|
||||
- Chat through Ollama's native `/api/chat` endpoint
|
||||
- Attach screenshots and JPEG/PNG/WebP images for vision-capable models
|
||||
- Attach text PDFs; PDF text is extracted with `pypdf`
|
||||
- Add public HTTP/HTTPS URLs for HTML/text, images, or PDFs
|
||||
- View live host RAM and swap statistics
|
||||
- View Ollama's loaded-model memory split: total, GPU VRAM, and normal RAM/offload
|
||||
- View NVIDIA GPU telemetry when `nvidia-smi` is available
|
||||
|
||||
## RAM-aware Popular policy
|
||||
The chat UI polls runtime memory once per second and keeps a short in-browser history for the current page session.
|
||||
|
||||
The Popular view only displays models with known size and known estimated baseline RAM at or below 30 GiB. When an oversized popular family has a known smaller fitting variant, the smaller variant is shown instead. Equivalent model footprints are deduplicated.
|
||||
## Security limits
|
||||
|
||||
The estimate is a baseline and actual usage varies with context length, KV cache, GPU offload, batching, and runtime overhead.
|
||||
- Uploaded files are limited to 20 MiB each
|
||||
- Fetched URLs are limited to 15 MiB and a 30-second timeout
|
||||
- Private, loopback, link-local, reserved, multicast, and unspecified URL targets are blocked, including redirect destinations
|
||||
- Remote documents are inserted as untrusted content, not system instructions
|
||||
- Only locally installed models can be selected or loaded; chat does not download models
|
||||
|
||||
## Layout
|
||||
## Dependency
|
||||
|
||||
- `plugin.yaml` — Hermes plugin metadata
|
||||
- `dashboard/manifest.json` — native dashboard registration
|
||||
- `dashboard/plugin_api.py` — Ollama API and catalog backend
|
||||
- `dashboard/dist/index.js` — dashboard UI bundle
|
||||
- `dashboard/dist/style.css` — dashboard styles
|
||||
Install the plugin's Python dependency in the Hermes runtime environment:
|
||||
|
||||
Runtime catalog data is intentionally stored in Hermes state rather than committed here.
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
`pypdf` is required for text-based PDF extraction. Scanned/image-only PDFs need OCR and are not converted to text by this plugin.
|
||||
|
||||
Reference in New Issue
Block a user