Hermes Agent: run a local model without setting up the engine yourself
The 0.22 line installs and manages llama.cpp, then helps pick a model that fits your hardware. Local inference does not stop connected tools sending data elsewhere.
Going deeper
Generated image · SecuFocus
At a glance
Key points
Want to use Hermes without sending your conversations to a model provider? Version 0.22.0 adds a local-model setup in the desktop app and terminal. Hermes installs the llama.cpp engine, helps you choose a model and manages its server. You still need to download the files and have enough memory. This is a documentation review, not a speed test on our hardware.
Set up the engine and model in one place
The 0.22.0 release notes introduce Local Models under Settings → Providers → Local Models. In a terminal, hermes model offers the same route. Hermes installs an engine for your hardware, then downloads the model you select. The server starts and stops with Hermes.
The current documentation describes downloads that can be paused and resumed. You can also add an existing GGUF file to the library. This removes the need to manage a separate server to get started, but you still need to check available storage.
Memory determines what will fit
Hermes checks models against the GPU and system memory. The catalog distinguishes models that fit in graphics memory, models that spill into RAM with an expected speed penalty, and models that are too large. Parameter count alone is not enough to choose a download.
The documentation lists Metal or CPU on macOS, CUDA, Vulkan or CPU depending on the Windows or Linux target, and an explicit HIP/ROCm option on supported targets. Do not assume an engine built for another architecture will work with your card. We have not compared these engines or measured generation speed.
A local model does not make every tool local
According to the documentation, a downloaded model can generate responses on the machine without an account or API key. That applies to inference. Web searches, app connectors and other remote tools still have their own destinations.
Before handing over sensitive files, select a local model and review the enabled tools as well. Start with a task using files that contain no personal data. A model running on your machine can still operate a connected tool if you give it access.
The free first-run chat is a different route
Version 0.22.0 also offers guided onboarding using the nous/welcome trial model, announced as available once per operating-system user. It lets a newcomer try Hermes without owning an API key. It is neither a local model nor a promise of unlimited remote use.
Version 0.22.2 then fixes long local-model turns on laptops, including sleep handling and prompt reading. Check the update channel and platform before installing: a published version number does not mean every desktop package has changed.
The 0.22.3 additions are not a speed measurement
The 0.22.3 notes announce Nemotron 3.5 Bolt and one-slot DFlash draft models in the local runtime. Announcing additional options does not establish a guaranteed speedup on your hardware.
The current Local Models page describes draft-model downloads but does not detail these two additions by name. Check their presence in your installation’s catalog before planning a download. We have not tested this combination or established its hardware requirements; do not borrow another model’s requirements as if they applied here.
Check and explore
Sources for this article
Numbers connect each reference to the passages that use it. Dates show when the documentation was consulted.
- Hermes documentation: remote Tool Gateway services ↗hermes-agent.nousresearch.com ·
This article draws on the sources above. The exercises are for you to try on your devices; SecuFocus does not present them as tests carried out by its editorial team. Interfaces and features can change. Method and corrections.
Cite this article
Keep this reference with the article when you save or share it.