feat: add primary model validation harness
This commit is contained in:
@@ -17,8 +17,10 @@ Native-like Hermes dashboard plugin for local Ollama model management and chat.
|
||||
- Paste images directly into the composer and drag/drop images, PDFs, and text files
|
||||
- Streamed Ollama responses with a real Stop action that cancels the active request
|
||||
- Minimized-by-default expandable thinking/progress details with live stage, elapsed time, event, and character counters
|
||||
- Validation harness mode: choose one primary model and two or more independent validator models; validators review the primary draft and the primary model compiles one final answer
|
||||
|
||||
The chat supports two modes. With one selected model, it sends a normal direct request. With one primary model and at least two validator models selected, the plugin runs a validation harness: the primary creates a draft, validators independently review the request and draft in parallel, and the primary compiles one final user-facing answer from the draft and validation reports. Validator reports are returned as supporting evidence, while only the compiled primary response is persisted and displayed as the answer.
|
||||
|
||||
The chat transcript and selected model are persisted in the Hermes server's SQLite database at `~/.hermes/ollama-manager/chat.sqlite3`, so conversations can be listed and resumed from another browser or after a dashboard restart. Use **New conversation** to start a separate thread and **Clear chat** to delete the selected shared conversation. Uploaded files remain temporary; attachment names/types/URLs are retained as metadata, not raw file contents.
|
||||
|
||||
## Performance metrics
|
||||
|
||||
@@ -93,7 +95,7 @@ When Ollama accepts a load request but evicts one model while starting another,
|
||||
While Ollama is starting a runner, the highlighted runtime chart displays an animated **Loading into Ollama memory** state with the selected model names, current stage, and elapsed time. The runtime panel separately reports Ollama resident model-weight bytes and the estimated target weight bytes. This is separate from host `MemAvailable`: CPU-mapped model files may appear as Linux file cache rather than ordinary process RAM usage.
|
||||
|
||||
## Multi-model loading and resident state
|
||||
The model pool now verifies every load request against Ollama `/api/ps` before reporting success. The UI shows an in-progress loading message, then reports which models are actually resident and which Ollama evicted. Resident models are highlighted in the pool with a green loaded state. After a browser refresh, resident models repopulate the pool selection and the **Models for this answer** selector, allowing multiple loaded models to be selected for parallel chat.
|
||||
The model pool now verifies every load request against Ollama `/api/ps` before reporting success. The UI shows an in-progress loading message, then reports which models are actually resident and which Ollama evicted. Resident models are highlighted in the pool with a green loaded state. After a browser refresh, resident models repopulate the pool selection and the answer-harness controls. Select one primary loaded model and two or more validator models; validators review the primary draft and the primary compiles one final answer.
|
||||
|
||||
Ollama still controls the physical resident-model limit. If it cannot keep all requested models at once because of its scheduler, GPU policy, context allocation, or available memory, the plugin reports the non-resident names instead of claiming they were permanently loaded. Increasing that limit requires changing the Ollama service configuration; the plugin does not silently alter or restart the Ollama service.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user