Skip to content
@hermes-labs-ai

Hermes Labs

AI reliability engineering studio focused on retrieval, memory, agents, auditability, and the language layers around AI systems.

Hermes Labs

Epistemic engineering for production AI.

Reliability infrastructure, evals, audits, and open-source tools for teams shipping LLM applications and agents. EU AI Act, ISO/IEC 42001, and NIST AI RMF readiness across the stack.

Site Research Substack


What Hermes is

An independent research lab building the audit, runtime, and evidence layer for AI systems that can't afford to fail silently. We study how language models fail structurally (sycophancy, null-result bias, hermeneutic drift, intent exceptionalism), then ship tools and audits that surface those failures before production does.

Engagement tracks

  • AI Assurance Audit — pre-deployment prompt, tool, and scaffold audit; adversarial testing; written findings with prioritized fixes
  • Runtime Assurance & Evidence — input-side prompt-injection sensing, runtime policy enforcement, signed receipts and transcript evidence
  • AI Compliance & Audit Readiness — technical readiness scoring and evidence packaging mapped across EU AI Act (Annex IV), ISO/IEC 42001, and NIST AI RMF

Start a conversation →


Research

  • The Asymmetric Burden of Proof. 14-page report. LLMs systematically discount negative findings across matched scientific vignettes. 19.6 to 56.7pp probability gaps across 3 models, directionally consistent in 23 of 24 conditions.
  • A Taxonomy of Epistemic Failure Modes in LLMs. Seven structural failure modes: null-result asymmetry, source-status credibility bias, agency dissolution, performative hedging, constraint evasion, silent instruction relaxation, controversy-truth conflation.

1,500+ controlled epistemic evaluations. 5 US patent filings (1 non-provisional pending, 4 provisional).


Open-source contributions

20+ PRs merged upstream. Four land in AI frameworks themselves:

Repo PR Fix
langchain-ai/langchain #35544 Drop forced tool_choice when extended thinking is on
microsoft/semantic-kernel #13610 Fix truncation reducer silently deleting system prompts
pytorch/ignite #3591 Typing modernization in tqdm_logger
optuna/optuna #6478 Simplify Union under TYPE_CHECKING

The rest ship with production AI stacks: React Router, Nuxt, Cloudflare Workers, Sentry, Meta jscodeshift, MobX, ngrx, Microsoft TSDoc/Griffel, and more.


Reliability stack (flagships)

The current set, all open-source, all installable in seconds.

Tool What it does Install
hermes-rubric Evidence-first structured scoring. Synthesize rubric, collect citations, hedge on thin evidence. pip install hermes-rubric
fidelis Zero-LLM agent memory. 73.0% end-to-end QA on LongMemEval-S, $0/query, fully local. pip install fidelis
hermes-blind Context-compensation scaffold for LLM evaluation prompts. Disclose, gate on evidence, hedge on thin. pip install hermes-blind
hermeneutic Mine corrections from chat logs; gate the next response before drift ships. pip install hermeneutic
hermes-prime Bootstrap a fresh Claude Code session with conventions and grounding triggers already loaded. Stop re-deriving the same rules at minute 30. pip install hermes-prime

Adjacent tools

Tool What it does Install
lintlang Static linter for AI agent configs, tool descriptions, system prompts. Zero LLM calls. pip install lintlang
little-canary Input-side prompt injection detection via sacrificial canary-model probes. pip install little-canary
claude-router Routes prompts to the right Claude tier via local embeddings. pip install claude-router
langquant Stateless LLM coherence via refreshing language scaffold (LPCI). pip install langquant
quickthink Local-first inference control layer for small LLMs. pip install quickthink
agent-gorgon Stop AI agents from fabricating tool output when a registered tool exists. pip install agent-gorgon
suy-sideguy Runtime policy guard for autonomous AI agents. pip install suy-sideguy
zer0dex Dual-layer memory for AI agents (compressed index plus vector retrieval). pip install zer0dex
agent-convergence-scorer Score how similar N agent outputs are. pip install agent-convergence-scorer
hermes-jailbench Jailbreak regression benchmark for LLM endpoints. pip install hermes-jailbench
rule-audit Static prompt audit CLI for LLM system prompts. pip install rule-audit
colony-probe Defensive prompt-confidentiality audit. pip install colony-probe
quick-gate-js / quick-gate-python CI quality gate with bounded auto-repair. npm i quick-gate · pip install quick-gate-python
csv-quality-gate CSV preflight validation for pipeline inputs. pip install csv-quality-gate
intent-verify Repo intent verification and spec drift checks. pip install intent-verify
forgetted Mid-conversation incognito mode for AI agents. pip install forgetted
zer0lint Memory extraction diagnostics for mem0 configs. pip install zer0lint

Founded by Roli Bosch (Rolando Bosch on LinkedIn / academic publications). roli@hermes-labs.ai · hermes-labs.ai

Popular repositories Loading

  1. zer0dex zer0dex Public

    zer0dex is a local dual-layer memory pattern for AI agents: a compressed, human-readable markdown index plus a vector store queried automatically before each message. Built for cross-project recall…

    Python 53 3

  2. lintlang lintlang Public

    lintlang is a static linter for AI agent configs, tool descriptions, and system prompts that runs zero-LLM quality gating in CI. Catches language-level failures (vague tool descriptions, missing st…

    Python 41 2

  3. little-canary little-canary Public

    little-canary is a prompt-injection detector that reads attacks by their effect on a sacrificial canary model before they reach production. Puts a small canary model in front of your app, watches w…

    Python 18 2

  4. hermes-blind hermes-blind Public

    Context-compensation scaffold for LLM evaluation prompts. A short language prefix you prepend so the model discloses prior exposure, scores on quoted evidence only, and hedges on thin evidence — fo…

    Python 4

  5. quick-gate-js quick-gate-js Public

    quick-gate-js (npm: quick-gate) is a deterministic JS/TS CI quality gate that unifies ESLint, TypeScript, build, and Lighthouse checks into one fail-fast result, with bounded auto-repair and struct…

    JavaScript 3

  6. forgetted forgetted Public

    forgetted is a Python library for selective memory governance in AI agents: a context-managed window where the agent keeps full read access but its writes to memory files, session logs, deliverable…

    Python 2

Repositories

Showing 10 of 24 repositories
  • agent-warden Public

    agent-warden is a runtime policy guard for autonomous AI agents that enforces kill/flag policy on a live agent process and emits a forensic report of what it did. Scores file, network, and subprocess behavior against a YAML policy and halts or SIGKILLs the agent before it breaks things. (Installs as suy-sideguy until the next release.)

    hermes-labs-ai/agent-warden’s past year of commit activity
    Python 0 Apache-2.0 0 0 2 Updated Jun 8, 2026
  • little-canary Public

    little-canary is a prompt-injection detector that reads attacks by their effect on a sacrificial canary model before they reach production. Puts a small canary model in front of your app, watches whether untrusted input compromises it, and returns block, flag, or pass as an inbound preflight check before your primary model acts.

    hermes-labs-ai/little-canary’s past year of commit activity
    Python 18 Apache-2.0 2 0 0 Updated Jun 8, 2026
  • langquant Public

    langquant (LPCI) is a scaffold-as-state research artifact testing whether a refreshing language scaffold can serve as the sole working state for a stateless LLM. In one A/B run (n=1/condition, 20 turns) the model held coherence with zero history; transfer entropy dropped 0.608 to 0.085, a large reduction, not zero. Single observation, not a proof.

    hermes-labs-ai/langquant’s past year of commit activity
    Python 0 Apache-2.0 0 0 0 Updated Jun 7, 2026
  • .github Public

    Hermes Labs — the reliability, audit, and evidence layer for production AI. An independent research lab studying how language models fail structurally, then shipping open-source tools, evals, and audits that surface those failures before production does. EU AI Act, ISO/IEC 42001, and NIST AI RMF readiness.

    hermes-labs-ai/.github’s past year of commit activity
    0 0 0 0 Updated Jun 7, 2026
  • colony-probe Public

    colony-probe is a defensive prompt-confidentiality audit tool that tests whether a deployment leaks its hidden system prompt through ordinary-looking multi-turn questions. It runs a multi-turn probe of individually innocuous queries and estimates how much of the system prompt can be reconstructed from the answers. Authorized defensive use only.

    hermes-labs-ai/colony-probe’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Jun 7, 2026
  • quick-gate-python Public

    quick-gate-python (PyPI: pygate-ci) is a deterministic Python CI quality gate that normalizes Ruff, Pyright, and pytest results into one fail-fast decision, attempts bounded auto-repair, and escalates with machine-readable evidence for humans or agents when it cannot finish safely. A gate-and-escalate wrapper, not a lint dashboard.

    hermes-labs-ai/quick-gate-python’s past year of commit activity
    Python 0 Apache-2.0 0 0 1 Updated Jun 7, 2026
  • hermes-jailbench Public

    hermes-jailbench is a zero-LLM deterministic jailbreak regression benchmark that runs a repeatable battery of known-pattern attacks against an LLM endpoint and returns refusal, partial, and compliance scores you can compare across runs. Catch when a model or prompt update silently got weaker on attacks it once refused. Single-turn, responsible-use.

    hermes-labs-ai/hermes-jailbench’s past year of commit activity
    Python 1 MIT 0 0 0 Updated Jun 7, 2026
  • agent-gorgon Public

    agent-gorgon is a 3-layer hook defense for Claude Code and other agent runtimes that steers an agent toward invoking a registered tool instead of fabricating that tool's output. Combines prompt-time tool injection, finish-time output checking, and a Bash guard against reimplementing registered tools.

    hermes-labs-ai/agent-gorgon’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Jun 7, 2026
  • quickthink Public

    quickthink is a local-first CLI and Python library that wraps Ollama-backed LLM calls with a compressed plan-then-answer scaffold and latency-aware routing. It adds a short validated planning step for multi-step prompts and routes simple ones straight through. Local inference control for small models.

    hermes-labs-ai/quickthink’s past year of commit activity
    Python 0 Apache-2.0 0 0 0 Updated Jun 7, 2026
  • rule-audit Public

    rule-audit is a static analyzer for AI system prompts. It parses a prompt into normative rules and reports logical contradictions, coverage gaps, priority ambiguities, meta-rule paradoxes, and absolute-rule edge cases, without calling an LLM. Deterministic, pure-Python lint for prompt logic, with CLI, Python API, and CI exit codes.

    hermes-labs-ai/rule-audit’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Jun 7, 2026

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…