Hermes-Ollama_Models/README.md

Hermes Ollama Models

Native-like Hermes dashboard plugin for local Ollama model management and chat.

Chat capabilities

• Select an installed Ollama model and load it into memory
• Chat through Ollama's native /api/chat endpoint
• Attach screenshots and JPEG/PNG/WebP images for vision-capable models
• Attach text PDFs; PDF text is extracted with pypdf
• Add public HTTP/HTTPS URLs for HTML/text, images, or PDFs
• View live host RAM and swap statistics
• View Ollama's loaded-model memory split: total, GPU VRAM, and normal RAM/offload
• View NVIDIA GPU telemetry when nvidia-smi is available
• Chat is the default view when the Ollama Models plugin opens
• Natural composer behavior: Enter sends; Shift+Enter creates a new line
• Paste images directly into the composer and drag/drop images, PDFs, and text files
• Streamed Ollama responses with a real Stop action that cancels the active request
• Minimized-by-default expandable thinking/progress details with live stage, elapsed time, event, and character counters

The chat transcript and selected model are persisted in the Hermes server's SQLite database at ~/.hermes/ollama-manager/chat.sqlite3, so conversations can be listed and resumed from another browser or after a dashboard restart. Use New conversation to start a separate thread and Clear chat to delete the selected shared conversation. Uploaded files remain temporary; attachment names/types/URLs are retained as metadata, not raw file contents.

Performance metrics

Each completed request records Ollama-provided counters and timing when available:

• Time to first token (TTFT)
• Total request latency
• Prompt/input token count and prompt tokens/sec
• Output token count and output tokens/sec
• Total tokens
• Ollama total/load/prompt-evaluation/generation durations
• Model, request status, request ID, and error details

The dashboard shows per-conversation metrics and rolling aggregates across the latest 100 requests. Values are shown as unavailable when Ollama does not provide them; the plugin does not estimate token counts.

Catalog and download filters

The Popular and Available downloads views only show models with known size and expected RAM estimates that fit the running host's installed RAM, detected from /proc/meminfo and reported in the API filter metadata. The limit is calculated at runtime, so it follows RAM upgrades or downgrades without a hard-coded threshold. Available downloads also includes known fitting variants from installed model families.

The Available downloads controls support:

• Dense or MoE model type
• Abilities such as completion, thinking, tools, vision, audio, and video when supplied by the catalog
• Organization by upstream popularity, newest (modified_at), smallest size, largest size, or name
• Search across model names, families, strengths, and capabilities

Popularity and date ordering use upstream metadata only; the plugin does not invent popularity, dates, RAM requirements, or token metrics.

Installation

This plugin is installable from the Hermes dashboard Plugin Section using the repository URL:

https://gitea.beyondcloud.solutions/dennii/Hermes-Ollama_Models.git

In Hermes Dashboard, open Plugins, choose Install from repository, enter the URL above, and install. The repository contains the root plugin.yaml, dashboard manifest, backend API, compiled frontend bundle, stylesheet, and an opt-in prerequisite declaration. On Linux, the Hermes installer will verify Ollama, install it with the official Ollama installer when missing, and install the plugin's pypdf dependency before committing the plugin into ~/.hermes/plugins/. Ollama installation requires the Hermes container to run as root, which is the expected configuration for a privileged ZimaOS deployment. If the container is not running as root, the plugin install stops without enabling a partially configured plugin. After installation or an update, restart only the Hermes dashboard service if requested by the installer.

The dashboard now includes a connection panel in the header. Enter an Ollama base URL, choose Local or Remote, and use Test before Save. Saved URLs are stored in Hermes plugin data with file permissions restricted to the Hermes account. Credential-bearing URLs are rejected.

The plugin probes the configured local endpoint and known Docker endpoints, and tests a configured remote endpoint. A successful connection reports the Ollama version and installed model count. The active local endpoint continues to drive chat, status, load, and runtime telemetry.

When both local and remote endpoints are online, downloading or re-downloading a model opens a destination chooser. The selected target is recorded on the job and the pull is sent to that endpoint. Remote downloads do not alter the local installed-model inventory.

The plugin supports both deployment types:

• Physical/Linux Hermes: Ollama defaults to http://localhost:11434. If Ollama is missing, the prerequisite installer can install it on a root-run host.
• Docker Hermes: Ollama should normally run as a separate service or on the Docker host. The plugin does not install Ollama inside the Hermes container. It first probes http://ollama:11434 and then http://host.docker.internal:11434. Set OLLAMA_HOST to the reachable Ollama base URL, without /v1, for example http://ollama:11434, http://host.docker.internal:11434, or the host address shown by your installation, such as http://<host-ip>:11434.

For host access on Linux Docker, add this to the Hermes service when needed:

extra_hosts:
  - "host.docker.internal:host-gateway"
environment:
  OLLAMA_HOST: http://host.docker.internal:11434

For Compose service-to-service networking, use the Ollama service name instead:

environment:
  OLLAMA_HOST: http://ollama:11434

The dashboard plugin API is mounted when the dashboard starts. Restart Hermes after installing or enabling the plugin; otherwise the plugin page can load while /api/plugins/ollama-manager/status still returns 404.

Security limits

• Uploaded files are limited to 20 MiB each
• Fetched URLs are limited to 15 MiB and a 30-second timeout
• Private, loopback, link-local, reserved, multicast, and unspecified URL targets are blocked, including redirect destinations
• Remote documents are inserted as untrusted content, not system instructions
• Only locally installed models can be selected or loaded; chat does not download models

Dependency

The Hermes Plugin Section installs the declared pypdf dependency automatically before the plugin is committed. For a manual checkout, install it with:

pip install -r requirements.txt

pypdf is required for text-based PDF extraction. Scanned/image-only PDFs need OCR and are not converted to text by this plugin.