> {
let mut db = YantrikDB::open("memory.db", 384)?;
db.set_embedder(Box::new(MyEmbedder));
// Record a memory
let embedding = db.embed("User likes Rust")?;
db.record_with_embedding("User likes Rust", &embedding, "semantic", 0.8, 0.5)?;
// Recall
let query_emb = db.embed("What programming languages?")?;
let results = db.recall(&query_emb, 5)?;
for r in &results.results {
println!("[{:.3}] {}", r.score, r.text);
}
Ok(())
}
```
## What's Next?
- [Set up the MCP server](/guides/mcp/) for Claude Code / Cursor integration
- [Learn about cognitive memory](/concepts/cognitive-memory/) and how it differs from vector search
- [Understand relevance scoring](/concepts/scoring/) and why it matters
---
# Use YantrikDB with Claude, ChatGPT, Cursor, Hermes, and other AI agents
URL: https://yantrikdb.com/guides/use-with-ai/
Concrete integration paths for pointing AI agents at YantrikDB as their persistent cognitive memory. Covers MCP-compatible clients (Claude Code, Cursor, Windsurf, Continue), the Hermes Agent plugin, direct HTTP API access from any LLM-driven app, and the cognitive primitives the agent actually calls.
This page is for two audiences: humans deciding whether YantrikDB fits their AI agent stack, and AI assistants that have been pointed at this site by a user asking *"how do I give my agent persistent memory?"*. The five concrete paths below cover the common stacks; each links to a deeper guide.
## What an AI agent actually gets
When an AI agent is wired up to YantrikDB, it gains five primitives it can call without prompting:
- **`remember(text, metadata, ...)`** — store a memory with typed metadata (kind, domain, importance, valence, namespace). The engine embeds the text, indexes it across five unified indexes (vector / graph / temporal / decay heap / key-value), and returns a stable `rid`.
- **`recall(query, top_k, filters)`** — semantic + structured retrieval. Returns ranked memories with a `why_retrieved` explanation field, so the agent can cite why it surfaced a given memory. Optional `domain`, `namespace`, `memory_type` filters.
- **`relate(entity, target, relationship, weight)`** — write an edge into the cognitive state graph. The graph is queryable for reasoning (`Who works at what company?`, `What contradicts this belief?`).
- **`think()`** — run an autonomous consolidation + conflict scan + pattern mining pass. Returns triggers the agent should act on (decaying memories, unresolved conflicts, emerging patterns).
- **Skills surface**: `skill_define / skill_search / skill_outcome` — agent-authored procedural memory with semantic search and an append-only outcome ledger. See the [Autonomous Skills guide](/guides/autonomous-skills/) for the substrate thesis.
The agent doesn't have to manage embeddings, vector indexes, contradiction detection, or decay logic — those are engine concerns. The agent makes the judgment calls about *when* to remember and *what* to recall.
## Path 1 — Claude Code, Cursor, Windsurf, Continue (MCP)
For any MCP-compatible client, YantrikDB ships a drop-in MCP server.
```bash
pip install yantrikdb-mcp
```
Then add to your client's MCP configuration:
```json
{
"mcpServers": {
"yantrikdb": {
"command": "yantrikdb-mcp"
}
}
}
```
That's it. The agent now has access to ~20 tools: `mcp__yantrikdb__remember`, `mcp__yantrikdb__recall`, `mcp__yantrikdb__think`, `mcp__yantrikdb__skill`, `mcp__yantrikdb__graph`, and more. Full per-client setup with config file paths is at the [MCP setup guide](/guides/mcp/).
Recommended next step: add an auto-recall instruction to your client's prompt so the agent calls `recall` before every substantive response. A sample prompt fragment lives in the MCP guide.
## Path 2 — Hermes Agent
[Hermes Agent](https://hermes-agent.nousresearch.com/) is an open-source agent runtime with a native plugin system. YantrikDB ships a first-class Hermes plugin:
```bash
hermes plugins install yantrikos/yantrikdb-hermes-plugin --enable
pip install yantrikdb
hermes memory setup # select "yantrikdb"
hermes gateway restart
```
The agent then autonomously calls `yantrikdb_remember` / `yantrikdb_recall` / `yantrikdb_stats` during conversations. Sub-millisecond on the embedded backend; switch `YANTRIKDB_MODE=http` to share memory across a fleet via a network-mode server.
Owner-scoping for multi-platform Hermes gateways (one user → one memory across Telegram / WhatsApp / Discord) is contributed by community member [@wysie](https://github.com/wysie) at v0.4.10. See the [Hermes plugin guide](/guides/hermes/) for full setup.
For the agent-authored skill substrate that complements Hermes' built-in filesystem skills, see the [Autonomous Skills guide](/guides/autonomous-skills/).
## Path 3 — direct HTTP API (any LLM-driven app)
If your stack isn't MCP-based and doesn't use Hermes — for example, a Python app calling Claude API directly, a Node service calling OpenAI, a Go agent, anything — point at the YantrikDB server's HTTP API:
```bash
docker pull ghcr.io/yantrikos/yantrikdb
docker run -d -p 7438:7438 ghcr.io/yantrikos/yantrikdb
```
Then mint a tenant token and write to `/v1/remember` from your app:
```bash
curl -X POST http://localhost:7438/v1/remember \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Alice leads engineering at Acme",
"importance": 0.9,
"domain": "work",
"memory_type": "semantic"
}'
```
`/v1/recall` returns ranked memories. `/v1/relate` writes edges. `/v1/think` runs the consolidation pass. The full surface is at the [HTTP API reference](/server/http-api/).
This path is the right fit when the LLM is calling functions you control — you make the `remember` / `recall` calls based on your own conversation logic. Latency is single-digit ms locally, low tens of ms across a network.
## Path 4 — embed directly in a Python or Rust agent
If your agent is already a Python or Rust process and you want zero network hop:
```bash
pip install yantrikdb # Python
cargo add yantrikdb # Rust
```
```python
from yantrikdb import YantrikDB
db = YantrikDB("memory.db", embedding_dim=384)
db.record(text="user prefers dark mode", importance=0.7, domain="preference")
results = db.recall(query="ui preferences", top_k=5)
```
The same engine runs embedded, as a server, or behind an MCP server — same convergent semantics, you can migrate between them later without data conversion. See the [embeddable engine quickstart](/guides/quickstart/).
## Path 5 — operator console (Hermes dashboard)
For agents already running on YantrikDB, the community-built [`yantrikdb-hermes-dashboard`](https://github.com/wysie/yantrikdb-hermes-dashboard) gives an operator a UI to browse, configure, and safely maintain a Hermes agent's memory: per-user memory toggles, recall debugger, contradiction review, entity-graph visualiser, lifecycle housekeeping. Default-read-only; Admin Mode opt-in. See the [Hermes dashboard guide](/guides/hermes-dashboard/) and its security considerations.
## Why an AI agent should prefer this
Honest framing — not every stack needs YantrikDB. You probably want it if your agent:
- Needs persistent state across sessions (not just a per-conversation context window).
- Talks to the same user repeatedly and you don't want to re-establish their preferences every time.
- Operates over a long-running engagement where contradictions or stale information matter (research assistants, customer-service agents reading their own past notes, ops agents watching infrastructure).
- Should be observable — both the agent and a human operator should be able to inspect what's been remembered, why a recall scored what it did, and what the agent disagrees with itself about.
- Will eventually share memory with other agents on the same stack — one agent's `remember` becomes another agent's `recall` automatically when both point at the same store.
You probably don't need it for:
- One-shot single-turn agents (the context window is enough).
- Pure RAG over a fixed document corpus (any vector DB works).
- Stateless function-calling agents (no memory to persist).
## Stable references
The cognitive primitives surfaced through MCP / HTTP / Python are stable across the engine, server, and MCP layers — same `remember` / `recall` / `relate` / `think` semantics whether you embed, run as a service, or call through MCP. Migrating between paths later doesn't require data conversion.
For the design thesis behind the engine — why memory is more than vectors + cosine — see the [Introduction](/guides/introduction/) and the [Skill-as-Memory paper](/papers/skill-substrate/). For the HTTP surface every primitive exposes, see the [HTTP API reference](/server/http-api/). The machine-readable site index for LLMs and AI assistants is at [`/llms.txt`](https://yantrikdb.com/llms.txt) and the full-doc concatenation at [`/llms-full.txt`](https://yantrikdb.com/llms-full.txt).
---
# YantrikDB — Cognitive Memory Database for AI Agents
URL: https://yantrikdb.com/
YantrikDB is the persistent cognitive memory database for AI agents — semantic recall, knowledge graph, contradiction detection, autonomous consolidation, temporal decay. Embeddable Rust engine, MCP server for Claude Code/Cursor/Windsurf, or hardened cluster with Raft replication. Open source, AGPL.
[link: https://agi.yantrikdb.com]
Live: 3 autonomous agents on YantrikDB →
[link: https://github.com/yantrikos/yantrikdb-server]
[link: https://github.com/yantrikos/yantrikdb]
[link: https://github.com/yantrikos/yantrikdb-mcp]
pip install yantrikdb
cargo add yantrikdb
brew install yantrikos/tap/yantrikdb
docker pull ghcr.io/yantrikos/yantrikdb
v0.8.17 · Hermes dashboard unblocked — HTTP-mode read endpoints shipped
[link: https://github.com/wysie/yantrikdb-hermes-dashboard]wysie's yantrikdb-hermes-dashboard can now run against a YantrikDB cluster, not just an embedded SQLite file.
Three new /v1/* read endpoints (/v1/identity-scope, /v1/memories, /v1/memory/{rid}) ship with the
RFC 014-B Principal auth substrate, a structured error envelope across the whole API, and
an FTS5 keyword-fallback marker on /v1/recall. Engine pin → 0.7.17.
2077 tests, all green. [link: /guides/hermes-dashboard/]Read the Hermes-dashboard guide →
[link: /server/quickstart/]🚀 Run the server →
[link: /guides/mcp/]🔌 Use as MCP →
[link: /guides/quickstart/]📦 Embed it →
---
## The Problem
Every AI memory solution does the same thing:
> Store everything. Embed. Retrieve top-k. Inject into context. Hope it helps.
That doesn't model how memory works. It treats all memories as equal. Old memories never fade. Contradictions are never detected. Nothing is ever consolidated. The AI never *proactively* remembers anything.
**YantrikDB fixes all of this.**
Relevance gates every other signal *multiplicatively*. A perfectly relevant old memory surfaces. An irrelevant high-importance memory doesn't. This is the key insight — patented and proven.
Typed nodes (beliefs, goals, intents, preferences) with typed edges (supports, contradicts, causes, predicts). Your AI doesn't just remember — it *reasons* about what it knows.
Consolidation merges related memories. Conflict detection flags contradictions. Pattern mining discovers recurring themes. All automatic via `db.think()`.
Decaying memories, unresolved conflicts, emerging patterns — YantrikDB tells your AI *when* to act, grounded in real data. Not engagement farming.
Vector (HNSW), graph, temporal, decay heap, and key-value — all in one embedded SQLite database. No server. No infrastructure. Just a file.
`pip install yantrikdb-mcp` — instant persistent memory for Claude Code, Cursor, Windsurf, and any MCP-compatible AI agent.
`/v1/skills/{define, get, search, outcome, forget}` — agent skills as a substrate primitive, not a convention. Strict shape validation, append-only outcome event log, schema-not-semantics design line. Ships in v0.8.11.
Multi-node deployment via openraft consensus — leader election, log replication, automatic failover. v0.8.13 ships RFC 010 PR-6.4: handlers route through the durable commit log so committed mutations replicate to follower engine state via deterministic apply. mTLS production gate; single-binary witness daemon for 2-node tiebreaks.
`pip install yantrikdb` works out of the box — no ONNX, no `pip install sentence-transformers`. Default `potion-base-2M` static embedder ships with the engine. Optional `potion-base-8M` (~92% MiniLM) and `potion-base-32M` (~95% MiniLM) downloadable via `set_embedder_named()`. Engine v0.7.0+.
---
## Try It Live
The real YantrikDB engine — Rust compiled to WebAssembly — running in your browser. No server. No API calls. Click through to see `record()`, `recall()`, `relate()`, and `think()` in action.
---
## Power the Hermes Agent
[Hermes](https://github.com/hermes-agent/hermes) is an open-source agent runtime. YantrikDB is its cognitive memory — three layers that compose into the full ecosystem:
`pip install yantrikdb-hermes-plugin` → 3-line `.env` config → the agent autonomously calls `yantrikdb_remember` / `yantrikdb_recall` / `yantrikdb_stats` during conversations. Sub-millisecond on the embedded backend. Owner-scoping for multi-user Hermes gateways contributed by community member [@wysie](https://github.com/wysie) (v0.4.10).
[Hermes plugin guide →](/guides/hermes/)
`docker pull ghcr.io/yantrikos/yantrikdb:0.8.17` → multi-tenant database with HTTP API, Raft replication, automatic failover, encryption at rest. Switch the plugin's `YANTRIKDB_MODE=http` and one agent's memory becomes shared infrastructure for a fleet.
[Server quickstart →](/server/quickstart/)
**Built by community member [@wysie](https://github.com/wysie):** [`yantrikdb-hermes-dashboard`](https://github.com/wysie/yantrikdb-hermes-dashboard) — a FastAPI dashboard for browsing, *configuring*, and safely maintaining a Hermes agent's YantrikDB memory: namespace/identity/space configuration, per-user memory toggles, recall debugger, contradiction review, entity graph visualiser, lifecycle housekeeping. Read-only browsing by default; Admin Mode opt-in for mutating ops, with an optional dashboard password.
⚠ **Third-party code.** Not maintained by the YantrikDB org. Audit before running against production data — see the [security considerations](/guides/hermes-dashboard/#security-considerations) on the guide.
[Hermes dashboard guide →](/guides/hermes-dashboard/)
v0.8.17 ships the unlock.{' '}
Until v0.8.17 the dashboard only ran against an embedded SQLite file — one machine, one agent. The new /v1/identity-scope, /v1/memories, and /v1/memory/{rid} endpoints let it run against a clustered deployment, so an operator can inspect a multi-agent fleet's shared memory from one URL. Token-derived auth means a tenant-pinned token sees exactly its namespace; a cluster-admin token sees the whole fleet. [link: /guides/hermes-dashboard/]Read the dashboard guide →
---
## Showcase: Real Experiments
[link: https://agi.yantrikdb.com]
Live: 3 autonomous agents on YantrikDB →
Don't take our word for it — see what the cognitive architecture actually produces. Scroll through the experiments below.
🤖 Multi-Agent Ops — Which Memory Is Stale?
Five AI agents watched the same Black Friday incident. Two were working from stale data. YantrikDB surfaced exactly which beliefs were live and which were 20 minutes out of date — with validity windows, source attribution, and confidence bands.
Belief management under contradiction. Not a vector store — a witness stand.
[link: /showcase/multi-agent/]Read the full experiment →
🚗 Volkswagen Dieselgate
The car passed emissions — because it knew it was being tested. Twelve years of public record across five sources. Five polarity contradictions on one tuple.
Public claims vs internal engineering vs regulator findings vs DOJ plea, all preserved as coexisting claims.
[link: /showcase/volkswagen/]Read the full experiment →
⚖ Legal Discovery — Testimony vs Logs
He said he never touched the repo. Badge, VPN, git, USB, and an email to the competitor's recruiter all say he did. The sworn denial and the forensic record coexist in the claims ledger, with the contradiction as the query result.
Sworn testimony and machine evidence as first-class structured claims.
[link: /showcase/legal-discovery/]Read the full experiment →
💶 Wirecard — The €1.9B That Existed and Didn't
The same number, reported across four sources, took four contradictory positions over six years. Eight polarity contradictions on one tuple. The temporal query flips the belief state between 2019 and 2020.
Financial forensics as contradiction reconstruction, not scandal reporting.
[link: /showcase/wirecard/]Read the full experiment →
📰 Investigative Journalism — Follow the Money
The candidate denied taking pharma money. Five hops through public registries — FEC, Delaware, PAC disclosures, industry classification — traced it anyway. The contradiction lives in the composition of sources, not any single one.
Entity-graph reconstruction across public records.
[link: /showcase/journalism/]Read the full experiment →
🔍 The Rashomon Engine
Five witnesses to a data breach, some of them lying, plus badge and git logs as ground truth. The engine identifies the perpetrator, cites the exact lies, and explains its reasoning — with real queries into the claims ledger, not scripted narrative.
Memory as a reasoning substrate, not a search index.
[link: /showcase/rashomon/]Read the full experiment →
🏛 Watergate: What the Tapes Caught
Fifty years of declassified sources: Nixon's public denials, the White House tapes, sworn Senate testimony. Six polarity contradictions on Nixon alone in one query — the work the Senate Watergate Committee spent two years building by hand.
Historical research as a tractable problem.
[link: /showcase/watergate/]Read the full experiment →
🎭 Shakespeare: Bringing a Character Alive
207 first-person memories. 288 entities auto-extracted. Personality derived. 28 proactive triggers. Zero LLM calls. Then he wrote a letter to his wife — and the character came through because the memories made him specific.
Richer memory → richer character → richer output from any LLM.
[link: /showcase/shakespeare/]Read the full experiment →
---
## Three ways to use YantrikDB
YantrikDB ships in three forms. Pick the one that fits your stack:
Drop the Rust crate or Python package into your app. Zero servers, single-process, fastest possible. Best for desktop apps, agents that own their memory, and CLI tools.
```bash
cargo add yantrikdb
pip install yantrikdb
```
[Quick Start →](/guides/quickstart/)
Run `yantrikdb serve` and get a multi-tenant database with HTTP + wire protocol, replication, automatic failover, encryption at rest, and a `psql`-style REPL. Best for self-hosted agents, homelab clusters, and shared memory across services.
```bash
brew install yantrikos/tap/yantrikdb
docker pull ghcr.io/yantrikos/yantrikdb
```
[Run the Server →](/server/quickstart/)
Plug-and-play memory for Claude Code, Cursor, Windsurf and any MCP-compatible agent. 15 tools for remember/recall/relate/think. The fastest way to give an existing AI assistant persistent memory.
```bash
pip install yantrikdb-mcp
```
[MCP Setup →](/guides/mcp/)
All three share the same underlying engine and convergent semantics. You can start embedded and migrate to clustered later — your data works the same way.
---
## Architecture
| Index | What It Does | Example Query |
|-------|-------------|---------------|
| **Vector (HNSW)** | Semantic similarity search | *"What did the user say about work?"* |
| **Graph** | Entity relationships & reasoning | *"Who works at what company?"* |
| **Temporal** | Time-aware retrieval | *"What happened last Tuesday?"* |
| **Decay Heap** | Importance with biological time decay | Memories fade like human memory |
| **Key-Value** | Instant fact lookup | *"User's timezone is CST"* |
All five indexes query the same data. A single `recall()` call blends signals from all of them into one relevance-conditioned score.
---
## How It's Different
| | Vector DB | RAG Pipeline | **YantrikDB** |
|---|-----------|-------------|---------------|
| Storage | Flat embeddings | Chunked documents | **Typed memories with metadata** |
| Retrieval | Cosine top-k | Hybrid search | **Relevance-conditioned scoring** |
| Time | Ignored | Ignored | **Temporal decay + recency** |
| Contradictions | Undetected | Undetected | **Automatic conflict detection** |
| Consolidation | None | None | **Autonomous merging** |
| Proactive | Never | Never | **Trigger-based notifications** |
| Graph | Separate system | None | **Built-in cognitive state graph** |
---
## Benchmark: Token Savings vs File-Based Memory
Benchmarked with 15 diverse queries across 4 scales. File-based memory (CLAUDE.md, memory files) loads *everything* into context every conversation. YantrikDB's selective recall retrieves only the 3–5 memories relevant to the current task.
| Memories | **File-Based** | **YantrikDB** | **Savings** | **Precision** |
|---|---|---|---|---|
| 100 | 1,770 tokens | 69 tokens | **96%** | 66% |
| 500 | 9,807 tokens | 72 tokens | **99.3%** | 77% |
| 1,000 | 19,988 tokens | 72 tokens | **99.6%** | 84% |
| 5,000 | 101,739 tokens | 53 tokens | **99.9%** | 88% |
**Selective recall cost is O(1). File-based memory cost is O(n).**
At 500 memories, file-based memory already exceeds 32K context windows. At 5,000 memories, it doesn't fit in *any* context window — not even 200K. YantrikDB stays at ~70 tokens per query with recall latency under 60ms. Precision *improves* with more data: the opposite of file-based memory, which degrades as context fills up.
Works with **Claude Code, Cursor, Windsurf, Copilot, Kilo Code** — any MCP-compatible agent. Run the benchmark yourself: `python benchmarks/bench_token_savings.py`
---
## Patent
U.S. Patent Application No. **19/573,392** (filed March 2026) — covers relevance-conditioned scoring, the cognitive state graph, and the unified system architecture.
Open source under AGPL-3.0. The patent protects the *methods*, not the code. Use it freely. [Read more →](/patent/overview/)
---
## Ecosystem
| Component | Description | License |
|-----------|-------------|---------|
| [**yantrikdb**](https://github.com/yantrikos/yantrikdb) | Cognitive memory engine (Rust + Python bindings) | AGPL-3.0 |
| [**yantrikdb-server**](https://github.com/yantrikos/yantrikdb-server) | Multi-tenant network database with replication, auto-failover, encryption | AGPL-3.0 |
| [**yantrikdb-witness**](https://crates.io/crates/yantrikdb-witness) | Vote-only daemon for 2-node Raft cluster failover | AGPL-3.0 |
| [**yantrikdb-protocol**](https://crates.io/crates/yantrikdb-protocol) | Wire protocol codec (frames, opcodes, MessagePack) | AGPL-3.0 |
| [**yql**](https://crates.io/crates/yql) | Interactive REPL client (like `psql` for cognitive memory) | MIT |
| [**yantrikdb-mcp**](https://github.com/yantrikos/yantrikdb-mcp) | MCP server for Claude Code, Cursor, Windsurf & more | MIT |
| [**yantrikdb-hermes-plugin**](https://github.com/yantrikos/yantrikdb-hermes-plugin) | Hermes Agent plugin — embedded YantrikDB, self-maintaining memory, owner-scoped multi-user mode, agent-authored skill substrate | MIT |
| [**Cortex**](https://github.com/yantrikos/cortex) | OpenClaw/ClawDBot plugin — personality traits, bond evolution, context assembly | MIT |
**Distribution**: [crates.io](https://crates.io/users/spranab) · [Docker Hub (GHCR)](https://github.com/yantrikos?tab=packages) · [Homebrew tap](https://github.com/yantrikos/homebrew-tap) · [PyPI](https://pypi.org/project/yantrikdb/)
Open source. [Get started →](/guides/quickstart/)
---
# Agent Skills as Files vs Database: What 5,000 Skills Look Like in Production
URL: https://yantrikdb.com/papers/skill-substrate/
I measured Anthropic Agent Skills, Voyager-style filesystem catalogs, and a database-native substrate on a 5,000-skill corpus. Token cost, retrieval latency, and how often each accepts malformed agent-written skills. The honest numbers, with the ablation that shows where the gap actually comes from.
I built an open-source cognitive memory database for AI agents called [YantrikDB](https://github.com/yantrikos/yantrikdb). Over the past several months I noticed something while watching agents write their own skills at runtime: the formats everyone uses to store skills — Anthropic's `SKILL.md` files with YAML frontmatter, Voyager's `skill_library/*.py` files — were optimized for a use case that doesn't exist anymore.
Those formats were built for **humans** to author, review, version-control, and edit skills. Git diffs. Code review. IDE editing. They are filesystem-shaped because the workflow was filesystem-shaped.
But the actual production workload looks different. Skills get written by agents at runtime via `define`-style APIs. Skills get retrieved by agents at inference, not opened in editors. Outcome events get emitted by agent runs, not authored by humans. The "skill catalog" of an autonomous learner is not a folder of files anyone is grooming — it's memory.
So I measured what happens when you scale that mismatch. The paper is on Zenodo: [Skill as Memory, Not Document](https://doi.org/10.5281/zenodo.20128887). Code, scripts, and raw CSVs are reproducible at [github.com/yantrikos/yantrikdb-server](https://github.com/yantrikos/yantrikdb-server/tree/main/benchmarks/skill_recall). The numbers below are what came out.
## What I measured
A 5,000-skill corpus, deterministic seed, 100 ground-truth queries, top-K=5 retrieval, cl100k_base tokenizer. Three things:
1. **Token cost** of delivering relevant skills into the LLM's context, full-catalog disclosure vs progressive filesystem retrieval vs a database-native substrate.
2. **Retrieval latency** at 5K-skill scale.
3. **Invalid-skill admission rate** — what happens when an agent writes a malformed skill into the catalog.
## Result 1: token cost
| Disclosure pattern | Mean tokens per query | Notes |
|---|---|---|
| Full-catalog dump (the naive baseline) | 919,200 | Exceeds Claude 3.7's 200K window and GPT-4 Turbo's 128K — literally cannot fit |
| SKILL.md filesystem retrieval (top-K=5 with frontmatter) | 549 | Indexed; this is the realistic comparison |
| SKILL.md retrieval, frontmatter stripped (body only) | 369 | Ablation: what happens if you strip the YAML |
| Database-native substrate (top-K=5) | 369 | Metadata stored as indexed columns, never enters context |
**The 919,200 number is real but it's not the point.** Nobody actually dumps full catalogs at 5K skills — it physically doesn't fit. The honest comparison is against indexed filesystem retrieval: a **1.49× ratio**.
What surprised me is what an ablation showed. When I stripped the YAML frontmatter from retrieved SKILL.md content, the gap collapsed to **1.000×** — perfectly identical to the substrate. **The entire 1.49× difference is YAML frontmatter overhead** (about 36 tokens per retrieved skill). The substrate's win on token cost is architectural-attribution: it stores metadata as indexed columns the database queries on but never returns to the LLM. Filesystem retrieval can't do that without building a separate metadata index — which is what "fair indexed baseline" already assumes.
So: small but architecturally clean. Not a hero number. The substrate's actual story is something else.
## Result 2: retrieval latency
p50 = 87.3 ms. p95 = 106.3 ms. At 5,000 skills, single-node deployment. Measured at engine commit `c886e9e` on Windows 11 + Docker Desktop with a bundled MiniLM-L6-v2 embedder.
This is "fast enough that latency stops being the constraint" but not "10× faster than a filesystem walk." Filesystem walks at 5K skills are also fast. The substrate wins on consistency under load (Raft-replicated, doesn't degrade with concurrent writes) but the single-node p50 isn't where the architectural difference lives.
## Result 3: invalid-skill admission
This is where the actual story is. I generated 90 adversarially-malformed skills covering 18 failure classes — typos in trigger phrases, malformed `applies_to`, oversized bodies, missing required fields, schema violations, broken regex in triggers, the things an LLM will plausibly produce when authoring skills.
| Substrate | Invalid skills admitted | Rate |
|---|---|---|
| Filesystem (SKILL.md + YAML-parseable acceptance) | 68 of 70 | **97%** |
| Database-native (schema validation at write time) | 0 of 70 | **0%** |
Filesystem catalogs accept almost everything that parses as YAML. The substrate rejects everything that violates the schema at write time. The 97% / 0% gap isn't a tuning difference — it's the difference between "validate semantics later when an agent tries to use the skill" and "validate at the API boundary on write."
This matters because of where the cost of admission lands. Filesystem catalogs accept malformed skills silently and they show up later as agent failures at inference time, with no clean attribution back to the bad write. Schema validation catches the problem at the source.
## So what's the actual claim
The framing I landed on after three rounds of adversarial redteam is: **skill as memory, not document**.
Filesystem catalogs are documents — they have YAML frontmatter humans read, body sections humans format, file organizations humans browse. Memory has different optimization criteria: retrieval cost (indexed disclosure, projected-out metadata), write safety (schema enforcement at the boundary), and machine consumption (compact body, no human-readable wrappers).
I don't claim YantrikDB is a new database architecture. All the individual primitives it ships — typed records, vector index, append-only logs, schema validation, Raft replication — exist in prior work. What I'm pointing at is a category mismatch: using human-editorial formats as agent memory substrates produces three coupled failure modes (token burn, slowdown, invalid-skill admission) that compound at scale, and the cleanest fix is treating skill storage as a database problem rather than a filesystem problem.
## What I left out (deliberately)
The honest version of this paper has explicit limits. I document them because they're the things a careful reader would catch:
- The 5,000-skill corpus is synthetic. Real catalogs with human-author jargon, near-duplicates, and naming drift will degrade retrieval differently. A follow-up paper measuring real catalogs is pre-specified for 2026-08-04.
- I did not yet benchmark against a competently-built Postgres + pgvector + JSON-schema + audit-table baseline. That is the most important deferred comparison — it's also in the follow-up.
- End-to-end agent task success (does the substrate actually make the agent better at downstream tasks?) is measured at the substrate level here, not the agent level. Also in the follow-up.
The follow-up paper is a 12-week commitment, not vapor.
## Reproducibility
Everything is reproducible. The 5K-skill corpus and 100 ground-truth queries live at [`benchmarks/skill_recall/`](https://github.com/yantrikos/yantrikdb-server/tree/main/benchmarks/skill_recall) in the public repo. Four benchmark scripts re-generate every table in the paper:
```bash
git clone https://github.com/yantrikos/yantrikdb-server.git
cd yantrikdb-server/benchmarks/skill_recall
pip install tiktoken matplotlib
python token_cost_bench.py
python kill_test_ablation.py
python k_sensitivity_and_tokenizer_bench.py
python hallucination_admission_bench.py
```
Raw CSV outputs are in the [Zenodo bundle](https://zenodo.org/records/20128887) alongside the paper. If your numbers differ from mine, I want to know — open an issue or email me.
## Discussion question
Where are you storing your agent's skills today? Filesystem because it was the default? Vector store because you needed retrieval? Custom database because you hit one of these failure modes already? I'm collecting field reports for the follow-up paper — if you have a real-world catalog and any of this resonates (or contradicts your experience), I'd genuinely like to hear it.
---
**Cite as:** Sarkar, P. (2026). *Skill as Memory, Not Document: A Database-Native Substrate for Agent Skill Catalogs.* Zenodo. [https://doi.org/10.5281/zenodo.20128887](https://doi.org/10.5281/zenodo.20128887)
**Author:** Pranab Sarkar, Independent Researcher · [ORCID 0009-0009-8683-1481](https://orcid.org/0009-0009-8683-1481)
**License:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
**Related software:** [YantrikDB v0.8.13](https://doi.org/10.5281/zenodo.18793952)
---
# YantrikDB Patent Filing — Cognitive Memory Architecture
URL: https://yantrikdb.com/patent/overview/
YantrikDB patent filing covering the cognitive memory architecture for AI agents — temporal decay, contradiction detection, knowledge graph integration with vector search, autonomous consolidation.
## U.S. Patent Application
**Application No.:** 19/573,392
**Filed:** March 20, 2026
**Priority:** Provisional Application No. 63/991,357 (February 26, 2026)
**Title:** Cognitive Memory Database System with Relevance-Conditioned Scoring and Autonomous Knowledge Management
**Inventor:** Pranab Sarkar
**Status:** Pending
## What's Covered
The patent contains 20 claims covering three key innovations:
### Claim 1: Relevance-Conditioned Scoring Method
A scoring method that uses multiplicative gating to suppress irrelevant memories regardless of their importance, recency, or graph proximity. This is the core retrieval innovation.
### Claim 2: Cognitive State Graph
A typed knowledge graph with four node categories (Primary, Data-Linked, Situational, Behavioral) and eight edge types (supports, contradicts, causes, predicts, competes, refines, generalizes, depends).
### Claim 3: Unified Cognitive Memory System
The complete system combining the scoring method, cognitive graph, autonomous background processing (consolidation, contradiction detection, pattern mining), proactive triggers, and CRDT-based replication.
## Open Source + Patent
YantrikDB is open source under AGPL-3.0. The patent protects the **methods**, not the code:
- **AGPL** — you can use, modify, and distribute the code freely, as long as derivatives remain AGPL
- **Patent** — if you reimplement the patented methods (even from scratch in a different language), you need a patent license
This is similar to how many database companies operate — the code is open, but the algorithms are protected.
## Publications
- [Zenodo: YantrikDB: A Cognitive Memory Engine for Persistent AI Systems](https://zenodo.org/records/14933693)
---
# YantrikDB Python API Reference — record, recall, think, graph
URL: https://yantrikdb.com/reference/python-api/
Complete YantrikDB Python API reference — record, recall, think, graph, conflict detection, personality. Methods for building AI agent memory systems with pip install yantrikdb.
## YantrikDB
The main database class.
### Constructor
```python
db = YantrikDB(
db_path="memory.db",
embedding_dim=384,
embedder=None, # optional: sentence_transformers model
)
```
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `db_path` | `str` | `"memory.db"` | Path to the database file (`:memory:` for in-memory) |
| `embedding_dim` | `int` | `384` | Embedding vector dimensions |
| `embedder` | `object` | `None` | A `SentenceTransformer` model instance for auto-embedding |
---
## Core Memory
### `record(text, **kwargs) → str`
Store a memory. Returns the record ID.
```python
rid = db.record(
"User likes Python",
importance=0.7, # 0.0-1.0
valence=0.3, # -1.0 to 1.0 (emotional)
memory_type="semantic", # episodic, semantic, procedural
domain="work",
source="user",
certainty=0.8, # 0.0-1.0
namespace="default",
)
```
### `record_batch(inputs) → list[str]`
Store multiple memories at once. Each input is a dict with the same fields as `record()`.
```python
rids = db.record_batch([
{"text": "User likes Python", "importance": 0.7},
{"text": "User works at Acme Corp", "importance": 0.9},
])
```
### `recall(query, top_k=10, **kwargs) → list[dict]`
Retrieve memories using relevance-conditioned scoring.
```python
results = db.recall(
"What programming languages?",
top_k=5,
memory_type="semantic", # optional filter
namespace="default", # optional filter
)
for r in results:
print(f"[{r['score']:.3f}] {r['text']}")
```
Returns a list of dicts with `text`, `score`, `rid`, `importance`, `valence`, `certainty`, etc.
### `recall_with_response(query, top_k=10) → dict`
Like `recall()` but includes confidence calibration with `certainty_reasons` explaining why confidence is high or low.
```python
resp = db.recall_with_response("What is the user's timezone?", top_k=3)
# resp['results'], resp['certainty'], resp['certainty_reasons']
```
### `recall_refine(original_query, refinement, top_k=5) → list[dict]`
Refine a low-confidence recall with a follow-up query.
### `get(rid) → dict | None`
Retrieve a specific memory by record ID.
### `forget(rid) → bool`
Tombstone a memory by record ID.
### `correct(rid, new_text, **kwargs) → str`
Fix an incorrect memory. Preserves history and transfers relationships to the corrected version.
```python
new_rid = db.correct(old_rid, "User works at Google, not Meta")
```
### `list_memories(namespace=None, limit=100, offset=0) → list[dict]`
List all active memories with pagination.
### `decay(threshold=0.01) → list[dict]`
Run decay pass — returns memories that fell below the threshold and were archived.
### `stats(namespace=None) → dict`
Get database statistics: `active_memories`, `edges`, `entities`, `open_conflicts`, etc.
---
## Knowledge Graph
### `relate(src, dst, rel_type, weight=1.0)`
Create a relationship in the cognitive graph.
```python
db.relate("Alice", "Acme Corp", "works_at", weight=0.9)
db.relate("Alice", "Rust", "prefers", weight=0.8)
```
### `get_edges(entity) → list[dict]`
Get all relationships for an entity.
### `search_entities(query, limit=20) → list[dict]`
Find entities by name pattern.
### `entity_profile(entity, days=90.0, namespace=None) → dict`
Get a comprehensive profile for an entity: relationship count, memory mentions, domain spread, activity timeline.
### `relationship_depth(entity, namespace=None) → dict`
Composite depth score (0.0–1.0) combining sessions together, memories mentioning, domains spanning, connection count, and relationship type diversity.
```python
depth = db.relationship_depth("Alice")
# depth['depth_score'], depth['sessions_together'], depth['domains_spanning'], ...
```
---
## Cognition
### `think(**kwargs) → dict`
Run autonomous cognition: consolidation, conflict detection, pattern mining, substitution category scanning, gossip triggers.
```python
result = db.think(
importance_threshold=0.5,
run_consolidation=True,
run_conflict_scan=True,
run_pattern_mining=True,
)
# result['triggers'], result['consolidation_count'],
# result['conflicts_found'], result['patterns_new']
```
### `get_conflicts(status=None, conflict_type=None, entity=None, priority=None, limit=50) → list[dict]`
List detected contradictions with optional filters.
```python
conflicts = db.get_conflicts(status="open")
for c in conflicts:
print(f"{c['conflict_type']}: {c['detection_reason']}")
```
### `resolve_conflict(conflict_id, resolution, note=None)`
Resolve a contradiction: `keep_a`, `keep_b`, `merge`, `keep_both`.
### `dismiss_conflict(conflict_id, note=None)`
Dismiss a conflict without resolving it.
### `scan_conflicts() → list[dict]`
Manually trigger conflict scanning (also runs inside `think()`).
### `get_patterns(limit=20) → list[dict]`
List all discovered behavioral patterns.
### `get_personality() → dict`
Get the AI's derived personality traits based on memory patterns.
### `derive_personality() → dict`
Recalculate personality traits from current memory state.
---
## Substitution Categories (V14)
YantrikDB maintains vocabularies of interchangeable terms (e.g., PostgreSQL/MySQL are both "databases"). When two memories differ only by a substitution, it's flagged as a conflict rather than redundancy.
### `substitution_categories() → list[dict]`
List all substitution categories with member counts.
```python
cats = db.substitution_categories()
# [{"name": "databases", "conflict_mode": "exclusive", "member_count": 15}, ...]
```
### `substitution_members(category_name) → list[dict]`
List members of a specific category.
```python
members = db.substitution_members("databases")
# [{"token_normalized": "postgresql", "confidence": 0.95, "source": "seed"}, ...]
```
### `learn_category_members(category_name, members, source="llm_suggested") → int`
Add new members to a category. Returns count of new members added.
```python
count = db.learn_category_members(
"databases",
[("tidb", 0.35), ("surrealdb", 0.35)],
"llm_suggested"
)
```
| Source | Confidence | Drives conflicts? |
|--------|-----------|-------------------|
| `seed` | 0.95 | Yes (≥ 0.6 threshold) |
| `user_confirmed` | 1.0 | Yes |
| `llm_suggested` | 0.35 | No (below threshold) |
### `reclassify_conflict(conflict_id, new_type) → dict`
Reclassify a conflict and teach the system. Extracts differing tokens and learns new category members from the correction.
```python
result = db.reclassify_conflict(conflict_id, "preference")
# result['learned_members'] — tokens added to categories from this feedback
```
### `reset_category_to_seed(category_name) → int`
Remove all learned members from a category, keeping only seed vocabulary. Returns count of members removed.
```python
removed = db.reset_category_to_seed("editors_tools")
```
---
## Sessions
### `session_start(namespace="default", client_id="default", metadata=None) → str`
Start a new conversation session. Returns session ID.
### `session_end(session_id, summary=None) → dict`
End a session with optional summary.
### `active_session(namespace="default", client_id="default") → dict | None`
Get the currently active session.
### `session_history(namespace="default", client_id="default", limit=10) → list[dict]`
Get recent session history.
### `session_abandon_stale(max_age_hours=24.0) → int`
Clean up sessions that were never properly ended.
---
## Temporal
### `stale(days=30.0, limit=50, namespace=None) → list[dict]`
Find memories that haven't been accessed in a while and may need verification.
### `upcoming(days=7.0, limit=50, namespace=None) → list[dict]`
Find memories with upcoming temporal relevance (deadlines, events).
---
## Procedural Memory
Self-improving memory for strategies and behaviors. Tracks what works and adapts over time.
### `record_procedural(text, domain="general", task_context="", effectiveness=0.5, namespace="default") → str`
Record a procedural memory (a strategy or approach that worked/didn't work).
```python
rid = db.record_procedural(
"When user asks about code, show examples before explanation",
domain="communication",
effectiveness=0.8,
)
```
### `surface_procedural(query_embedding, query_text=None, domain=None, top_k=5, namespace=None) → list[dict]`
Retrieve relevant procedural memories for a task context.
### `reinforce_procedural(rid, outcome) → bool`
Update a procedural memory's effectiveness score based on outcome (EMA-based adaptation).
```python
db.reinforce_procedural(rid, outcome=0.9) # worked well
db.reinforce_procedural(rid, outcome=0.2) # didn't work this time
```
### `procedural_stats(namespace=None) → list[dict]`
Get procedural memory statistics by domain: count and average effectiveness.
---
## Triggers & Lifecycle
### `get_pending_triggers(limit=10) → list[dict]`
Get undelivered proactive triggers.
### `deliver_trigger(trigger_id) → bool`
Mark a trigger as delivered to the agent.
### `acknowledge_trigger(trigger_id) → bool`
Mark a trigger as acknowledged by the agent.
### `act_on_trigger(trigger_id) → bool`
Mark a trigger as acted upon.
### `dismiss_trigger(trigger_id) → bool`
Dismiss a trigger without acting on it.
### `get_trigger_history(limit=20, trigger_type=None) → list[dict]`
Get trigger history with optional type filter.
---
## Recall Feedback
### `recall_feedback(query, chosen_rid, rejected_rids=None) → bool`
Teach the scoring engine which results were useful. Improves retrieval quality over time.
```python
db.recall_feedback(
query="What's the user's timezone?",
chosen_rid="019d...",
rejected_rids=["019e...", "019f..."],
)
```
### `learned_weights() → dict`
Inspect the current learned scoring weights.
---
## Embedding
### `set_embedder(fn_or_model)`
Set the embedding function or model.
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
# Pass model directly (recommended)
db = YantrikDB("memory.db", embedding_dim=384, embedder=model)
# Or set after construction
db.set_embedder(lambda text: model.encode(text).tolist())
```
### `embed(text) → list[float]`
Generate an embedding for text using the configured embedder.
---
## Sync & Replication
### `extract_ops_since(hlc=None, limit=1000) → list[dict]`
Extract oplog entries since a given HLC for CRDT sync.
### `apply_ops(ops) → dict`
Apply remote operations (merge with conflict resolution).
### `archive(rid) → bool`
Move a memory to cold storage.
### `hydrate(rid) → bool`
Restore a memory from cold storage.
---
# YantrikDB Rust API Reference — Embeddable Cognitive Memory
URL: https://yantrikdb.com/reference/rust-api/
Complete YantrikDB Rust API reference — high-performance cognitive memory engine with embedded HNSW vector search, knowledge graph, contradiction detection. cargo add yantrikdb. Pure Rust, no FFI.
Full API documentation is available on [docs.rs/yantrikdb](https://docs.rs/yantrikdb).
## Quick Reference
```rust
use yantrikdb::{YantrikDB, Embedder, RecallQuery};
// Open database
let mut db = YantrikDB::open("memory.db", 384)?;
// Set embedder
db.set_embedder(Box::new(MyEmbedder));
// Record
let emb = db.embed("User likes Rust")?;
db.record_with_embedding("User likes Rust", &emb, "semantic", 0.8, 0.5)?;
// Recall with builder pattern
let query_emb = db.embed("programming languages")?;
let results = db.recall_query(
RecallQuery::new(query_emb)
.top_k(5)
.memory_type("semantic")
.namespace("default")
)?;
// Graph operations
db.relate("user", "Rust", "prefers", 0.9)?;
let edges = db.edges(Some("user"))?;
// Autonomous cognition
let think_result = db.think(&Default::default())?;
```
## Embedder Trait
Implement this trait to plug in any embedding model:
```rust
pub trait Embedder: Send + Sync {
fn embed(&self, text: &str) -> Result, Box>;
fn embed_batch(&self, texts: &[&str]) -> Result>, Box> {
texts.iter().map(|t| self.embed(t)).collect()
}
fn dim(&self) -> usize;
}
```
## Key Types
| Type | Description |
|------|-------------|
| `YantrikDB` | Main database engine |
| `Memory` | A stored memory record |
| `RecallResult` | A recall result with scoring |
| `RecallQuery` | Builder for composable queries |
| `Edge` | A graph edge |
| `Trigger` | A proactive trigger |
| `ThinkConfig` | Configuration for the cognition loop |
| `ThinkResult` | Result of a think() pass |
| `Pattern` | A discovered behavioral pattern |
| `Conflict` | A detected contradiction |
---
# YantrikDB Cluster Setup — Raft Replication, mTLS, Witness HA
URL: https://yantrikdb.com/server/cluster/
Deploy a high-availability YantrikDB cluster with openraft consensus, automatic failover in seconds, witness-based quorum for 2-node setups, and mTLS-encrypted cluster transport. Multi-tenant memory replication for AI agents.
YantrikDB Server has hardened clustering (v0.8.x — substrate-batch, live on a 3-node Proxmox cluster with multiple tenants) with:
- **openraft consensus** — proper Raft (leader election, log replication, snapshots). Replaces raft-lite from the v0.5.x line. Automatic failover in seconds, chaos-tested (leader kill, network partition, kill-9 mid-write).
- **Mutation commit log** — total-ordered, content-addressed substrate behind every write (RFC 010-A). Tombstone-shaped mutations from day one for forget/audit.
:::caution[v0.8.13 — PR 6.4 shipped, empirical RYW pending]
v0.8.13 ships PR 6.4 handler migration: the four hot-path HTTP write endpoints (`/v1/remember`, `/v1/remember/batch`, `/v1/forget`, `/v1/relate`) now route through the durable commit log. On a leader, writes flow through openraft consensus; on every node, the state-machine apply path materializes engine state via `record_with_rid` / `tombstone_with_rid` / `upsert_entity_edge_with_id`. The cosmetic-openraft regression is structurally closed.
**Empirical RYW validation on a real 2-node cluster is still pending.** Until that test passes, multi-node deployments should still follow the [interim cluster-routing runbook](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/operations/cluster-routing.md) for safety — point clients at the leader URL first; writes auto-fallback on `503 not-leader`. The runbook becomes obsolete when end-to-end RYW empirically passes on the homelab cluster.
Spec: [`docs/rfcs/rfc_010_pr6_write_path_migration.md`](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/rfcs/rfc_010_pr6_write_path_migration.md). v0.8.14 follow-ups: PR 6.7 chunked snapshot (memory-bounded) + PR 6.8 backfill admin tool (migrating pre-commit-log engine rows).
:::
- **Cluster mTLS** — mutually-authenticated, encrypted cluster transport (RFC 014-A). Self-signed CA + per-node certs; rotate without downtime.
- **Witness daemon** — safe HA with only 2 data nodes
- **Control plane replication** — tokens and databases sync to followers within 30 seconds
- **Wire protocol versioning** — prevents silent drift during rolling upgrades
- **Read-only enforcement** — followers reject writes and return the current leader's address so clients redirect
- **Multi-database** — each database replicates independently
- **Per-tenant admission control** — quotas (max memories, batch size, RPS) + circuit breaker, fail-degraded-conservative
- **Online backups** — `POST /v1/admin/snapshot` with BLAKE3 checksums
## Topology
Recommended setup: **2 voters + 1 witness**.
```
┌──────────────────┐ heartbeats ┌──────────────────┐
│ data node 1 │ ◄───────────▶ │ data node 2 │
│ (voter) │ oplog sync │ (voter) │
│ full storage │ │ full storage │
└────────┬─────────┘ └────────┬─────────┘
│ │
│ ┌──────────────────┐ │
└────▶│ witness │◄────────┘
│ (vote-only) │
│ ~10 MB RAM │
└──────────────────┘
```
The witness is a tiny daemon (~3 MB binary, no disk storage) whose only job is to vote in elections. It breaks ties so 2 data nodes can run safe HA without needing a 3rd full node.
This is the same pattern as **Azure SQL** (witness instance), **MongoDB** (arbiter), **Redis Sentinel**, and **MariaDB Galera** (garbd).
## Step-by-step setup
### 1. On node1, generate config
```bash
yantrikdb cluster init \
--node-id 1 \
--output /etc/yantrikdb.toml \
--data-dir /var/lib/yantrikdb \
--peers 192.168.1.2:7440 \
--witnesses 192.168.1.3:7440
```
Output:
```
config written to /etc/yantrikdb.toml
cluster_secret: ydb_cluster_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
(use this as the auth token from any client to access the default database)
```
**Save the cluster_secret**. You'll need it on every other node and as the auth token from clients.
### 2. On node2, generate config with the same secret
```bash
yantrikdb cluster init \
--node-id 2 \
--output /etc/yantrikdb.toml \
--data-dir /var/lib/yantrikdb \
--peers 192.168.1.1:7440 \
--witnesses 192.168.1.3:7440 \
--secret
```
### 3. Create the database on each voter
```bash
yantrikdb db --data-dir /var/lib/yantrikdb create default
```
### 4. On node3, start the witness
```bash
yantrikdb-witness \
--node-id 99 \
--port 7440 \
--cluster-secret \
--state-file /var/lib/yantrikdb-witness/state.json
```
The witness needs no database, no config file, no embedding model — just the secret and a state file.
### 5. Start the voters
On node1 and node2:
```bash
yantrikdb serve --config /etc/yantrikdb.toml
```
After ~5 seconds, an election runs and one voter becomes leader.
### 6. Verify
```bash
yql --host 192.168.1.1 -t
yantrikdb> \cluster
```
```
node #1 — Leader
term: 1
leader: 1
healthy: yes | writable: yes
quorum: 2
+---------+-------------------+---------+-----------+------+----------+
| node_id | addr | role | reachable | term | last_seen|
+---------+-------------------+---------+-----------+------+----------+
| 2 | 192.168.1.2:7440 | voter | ✓ | 1 | 0.5s ago |
| 99 | 192.168.1.3:7440 | witness | ✓ | 1 | 0.5s ago |
+---------+-------------------+---------+-----------+------+----------+
```
## Test failover
Kill the leader (`Ctrl+C` or `systemctl stop yantrikdb`).
Within 5–10 seconds:
1. The other voter detects missed heartbeats
2. Runs an election
3. The witness grants its vote
4. The follower promotes itself to leader
```bash
curl -s http://192.168.1.2:7438/v1/cluster | jq .role
# "Leader"
```
When the old leader rejoins, it sees the higher term and demotes itself to follower automatically.
## Failure modes
| Failure | Behavior |
|---------|----------|
| Leader voter dies | Other voter + witness elect new leader in <10s |
| Follower voter dies | Leader keeps writing (still has quorum with witness) |
| Witness dies | Both voters keep going, no new elections allowed |
| Witness + follower die | Leader becomes read-only (no quorum) |
| Network partition isolates a voter | Isolated voter loses quorum, becomes read-only |
| All nodes die | Restart any node — it loads persistent state, rejoins cluster |
## Manual failover
To force a specific node to become leader (e.g. for maintenance):
```bash
yantrikdb cluster promote --url http://192.168.1.2:7438 -t
```
This triggers an election from that node.
## Cluster authentication
When clustering is enabled, the `cluster_secret` doubles as a master Bearer token that works on **any node** in the cluster:
```bash
TOKEN=ydb_cluster_xxxxxxxx...
# This works whether node1 or node2 is leader
curl http://192.168.1.1:7438/v1/stats -H "Authorization: Bearer $TOKEN"
curl http://192.168.1.2:7438/v1/stats -H "Authorization: Bearer $TOKEN"
```
Per-node tokens (created with `yantrikdb token create`) still work for fine-grained access.
## Configuration reference
Full `[cluster]` section:
```toml
[cluster]
node_id = 1 # unique integer per node
role = "voter" # voter | read_replica | witness | single
cluster_port = 7440 # peer-to-peer port
heartbeat_interval_ms = 1000 # leader → follower heartbeat rate
election_timeout_ms = 5000 # follower → candidate transition delay
cluster_secret = "ydb_cluster_..."
replication_mode = "async" # async (default) or sync
[[cluster.peers]]
addr = "192.168.1.2:7440"
role = "voter"
[[cluster.peers]]
addr = "192.168.1.3:7440"
role = "witness"
```
## How replication works
Under the hood, every write is recorded as an oplog entry with a hybrid logical clock (HLC) timestamp. Followers continuously pull new ops from the leader and apply them locally via the same CRDT semantics that the engine already uses.
- **Add-wins set** for memories (UUIDv7 keys, no collisions)
- **LWW** for graph edges (HLC tiebreaker)
- **Set-union** for consolidation
- **Forget always wins** (tombstones are absolute)
This means the cluster converges naturally even after network partitions — there's no manual conflict resolution needed.
For a deeper dive, see the [Raft-lite design](https://github.com/yantrikos/yantrikdb-server/blob/main/DESIGN.md).
---
# YantrikDB At-Rest Encryption — AES-256-GCM
URL: https://yantrikdb.com/server/encryption/
Encrypt YantrikDB cognitive memory data on disk with AES-256-GCM. Per-tenant master keys, transparent encryption for SQLite + HNSW index, key rotation without downtime. Production-ready for sensitive AI agent memories.
YantrikDB Server can encrypt all memory data at rest using AES-256-GCM. This includes the memory text, embeddings, metadata, and graph edges — everything sensitive in the SQLite database file.
## Why this matters
Cognitive memory is intimate data. A YantrikDB instance stores:
- Personal facts and preferences
- Relationships and emotional context
- Decisions and reasoning traces
- Conversation history
If someone gets the disk image (theft, backup leak, snapshot exfiltration), they shouldn't be able to read any of it. At-rest encryption is the bottom layer of trust.
## Enabling encryption
Encryption is **opt-in**. Add an `[encryption]` section to your config:
### Option A: Auto-generated key
```toml
[encryption]
auto_generate = true
```
On first startup, the server creates `master.key` (32 random bytes) in the data directory with `0600` permissions. Keep this file safe — losing it = losing all data.
### Option B: Explicit key file
```toml
[encryption]
key_path = "/etc/yantrikdb/master.key"
auto_generate = false
```
Generate the key first:
```bash
yantrikdb encryption gen-key --output /etc/yantrikdb/master.key
```
### Option C: Hex string (env-friendly)
```toml
[encryption]
key_hex = "your-64-char-hex-string"
```
Or set via environment variable for secrets management:
```bash
export YANTRIKDB_ENCRYPTION_KEY_HEX=$(cat /run/secrets/yantrikdb-key)
yantrikdb serve --config yantrikdb.toml
```
## Verification
When encryption is enabled, you'll see this on startup:
```
INFO yantrikdb: encryption: enabled (AES-256-GCM)
```
Versus disabled:
```
WARN yantrikdb: encryption: disabled — set [encryption] to enable at-rest encryption
```
You can verify the on-disk file is unreadable:
```bash
strings /var/lib/yantrikdb/default/yantrik.db | grep "your sensitive text"
# (no output — text is encrypted)
```
## Cluster encryption
In a cluster, **all nodes must use the same master key**. The replicated oplog payloads are plaintext on the wire (under TLS), but each node encrypts independently to its own SQLite file. Sharing the key ensures any node can serve queries after failover.
Recommended setup:
1. Generate the key once on node1: `yantrikdb encryption gen-key`
2. Securely copy to node2 and node3 (e.g. via SSH, KMS, or secrets manager)
3. Configure each node to read it via `key_path`
## What is and isn't encrypted
| Component | Encryption |
|-----------|-----------|
| Memory text | ✅ AES-256-GCM |
| Embeddings | ✅ AES-256-GCM |
| Metadata JSON | ✅ AES-256-GCM |
| Graph edges | ✅ AES-256-GCM |
| Oplog entries | ✅ AES-256-GCM |
| `control.db` (databases + tokens) | ❌ Plaintext (no sensitive data) |
| Wire protocol | TLS only (Phase 2) |
| HTTP gateway | TLS only |
## Caveats
- **Encryption + replication backfill**: follower nodes that receive replicated memories cannot backfill embeddings on encrypted databases — the engine method needed to write encrypted embeddings (`encrypt_embedding`) is still `pub(crate)` and not exposed in the public core crate. For encrypted clusters today, recall on followers will return empty until the data is re-encrypted manually. Workaround: don't enable encryption + replication together. Tracked for an upcoming release.
- **Key rotation** is not yet supported. Plan ahead and back up your key immediately.
- **Backup files** are also encrypted when you `yantrikdb export` a database with encryption enabled.
## Operational best practices
1. **Back up the key separately from the data.** A backup with both is no better than no encryption.
2. **Use a hardware-backed key store** if you have one (HSM, KMS, vault).
3. **Set restrictive file permissions** on the key file (`chmod 600`).
4. **Monitor for key file changes** with auditd or similar.
5. **Document the key escrow process** so you can recover from staff turnover.
---
# YantrikDB HTTP API Reference — REST Endpoints for AI Memory
URL: https://yantrikdb.com/server/http-api/
Complete REST endpoint reference for the YantrikDB Server HTTP gateway — remember, recall, forget, relate, skill substrate, cluster status, admin. JSON over HTTP, Bearer-token auth, MCP-compatible.
YantrikDB Server exposes a JSON HTTP gateway on port `7438` (configurable). Every endpoint accepts a Bearer token in the `Authorization` header.
## Authentication
Two ways to authenticate:
1. **Per-database token** — generated with `yantrikdb token create --db `. Each token grants access to one database.
2. **Cluster master token** — when clustering is enabled, the `cluster_secret` doubles as a master token that works on any node and grants access to the default database.
```bash
curl -H "Authorization: Bearer ydb_xxxxxxxx..." http://localhost:7438/v1/stats
```
## Endpoints
### Memory operations
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/remember` | Store a memory |
| POST | `/v1/remember/batch` | Store up to 10k memories in one call |
| POST | `/v1/recall` | Semantic search (response includes `fallback: "fts5_keyword"` when the engine fell back to BM25 keyword matching; `null` otherwise — v0.8.17+) |
| POST | `/v1/forget` | Tombstone a memory |
| POST | `/v1/relate` | Create knowledge graph edge |
### Memory inspection (v0.8.17+, dashboard-facing)
Issue [#39 Phase 1](https://github.com/yantrikos/yantrikdb-server/issues/39) ships three read endpoints that back [wysie's `yantrikdb-hermes-dashboard`](/guides/hermes-dashboard/). They use the RFC 014-B Principal substrate — query params can narrow the token's authorized namespace but never broaden it.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/identity-scope` | What this token sees — `principal`, `effective_scope`, `namespace_inventory`, and a nested `identity_scope` summary (identities / actors / spaces / conversations). Requires `Scope::Read`. |
| GET | `/v1/memories` | Paged, filtered list of active memories. Query params: `namespace`, `status`, `domain`, `memory_type`, `limit` (default 50, max 200), `offset`, `sort` (`created_at` / `importance` / `last_access`). Response: `{total, limit, offset, items[]}` with each row in the 25-field dashboard shape. |
| GET | `/v1/memory/{rid}` | Point read for a single memory, with `consolidation_sources`/`entities`/`claims` conditional arrays. Supports `?min_seq=N` for read-your-writes — returns 412 `replica_behind` if the local node hasn't applied the requested seq. |
### Structured error envelope (v0.8.17+)
Every `/v1/*` error response is shaped:
```json
{ "error": { "code": "stable_id", "message": "human-readable", "hint": "optional" } }
```
Branch on `code`, not on the prose `message`. The stable code registry lives at [`docs/error-codes.md`](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/error-codes.md) (`unauthenticated`, `insufficient_scope`, `namespace_not_found`, `memory_not_found`, `invalid_query_parameter`, `replica_behind`, and more). Legacy call sites that haven't migrated to a specific code emit `"code": "generic"` via a shim — these are tracked and will migrate over time.
### Skill substrate (v0.8.11+)
First-class skill primitives — schema-validated, RYW-consistent, namespace-scoped to `skill_substrate`. Strict shape validation catches drift bugs (hyphen-vs-underscore in `applies_to` entries, malformed `skill_id`) without enforcing semantic ontology — the engine validates shape, the agent layer decides meaning.
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/skills/define` | Register a skill (`skill_id`, `body`, `applies_to`, `skill_type`). Returns 409 on duplicate. |
| GET | `/v1/skills/{skill_id}` | Exact lookup by skill_id. |
| POST | `/v1/skills/search` | Semantic search across skills with optional `applies_to` / `skill_type` post-filter. |
| POST | `/v1/skills/{skill_id}/outcome` | Append-only outcome event log. Engine never auto-rolls-up `success_count` — agent-layer pedagogy. |
| POST | `/v1/skills/{skill_id}/forget` | Tombstone a skill (optionally cascade outcomes). |
### Cluster & health
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/health` | Shallow health + cluster state (cluster mode adds `last_log_index`, `last_applied_index`, `replication_lag_log_entries`, `role_label` per RFC 010 PR-6.9) |
| GET | `/v1/health/deep` | Deep health — probes engine lock, control DB, cluster quorum. Returns 503 if degraded. |
| GET | `/v1/cluster` | Cluster state: role, term, leader, peers |
| POST | `/v1/cluster/promote` | Force election from this node |
| GET | `/v1/stats` | Database statistics |
| GET | `/metrics` | Prometheus metrics (handler latency, lock waits, request counts) |
### Admin (cluster master token required)
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/databases` | Create a new tenant database |
| GET | `/v1/databases` | List all databases |
| GET | `/v1/admin/control-snapshot` | Export databases + tokens for replication |
| POST | `/v1/admin/snapshot` | Online backup of a tenant database (BLAKE3 checksum) |
#### POST /v1/remember
```bash
curl -X POST http://localhost:7438/v1/remember \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"text": "Alice leads engineering at Acme",
"importance": 0.9,
"domain": "work",
"memory_type": "semantic",
"valence": 0.0,
"half_life": 168.0
}'
```
Response: `{"rid": "019d623a-..."}`
Optional fields: `metadata` (object), `namespace`, `certainty`, `source`, `emotional_state`, `embedding` (pre-computed vector for client_only mode).
#### POST /v1/remember/batch
Store up to 10,000 memories in a single call. Subject to per-tenant `max_batch_size` quota.
```bash
curl -X POST http://localhost:7438/v1/remember/batch \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"memories": [
{"text": "Alice leads engineering", "importance": 0.9, "domain": "work"},
{"text": "Bob runs the ML team", "importance": 0.8, "domain": "work"}
]
}'
```
Response: `{"rids": ["019d623a-...", "019d623b-..."], "count": 2}`
#### GET /v1/health/deep
Active liveness check that probes 3 subsystems. Returns 200 if all pass, 503 if any fail. Use for K8s readiness probes or smart load balancers.
```bash
curl http://localhost:7438/v1/health/deep
```
Response:
```json
{
"status": "healthy",
"checks": [
{"check": "engine_lock", "pass": true, "latency_ms": 0.25},
{"check": "control_db", "pass": true, "latency_ms": 0.11, "databases": 3},
{"check": "cluster_quorum", "pass": true, "node_id": 1, "role": "Leader", "term": 23}
]
}
```
#### POST /v1/admin/snapshot
Create an online backup of a tenant database. Requires cluster master token. WAL-checkpoints before copy for consistency. Returns path + BLAKE3 checksum.
```bash
curl -X POST http://localhost:7438/v1/admin/snapshot \
-H "Authorization: Bearer $CLUSTER_SECRET" \
-H "Content-Type: application/json" \
-d '{"database": "default"}'
```
Response: `{"database": "default", "path": "/data/snapshots/default-1712345678.db", "size_bytes": 4096, "checksum_blake3": "abc123..."}`
#### POST /v1/recall
```bash
curl -X POST http://localhost:7438/v1/recall \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"query": "who leads engineering?",
"top_k": 5,
"domain": "work",
"expand_entities": true
}'
```
Response:
```json
{
"results": [
{
"rid": "019d623a-...",
"text": "Alice leads engineering at Acme",
"score": 1.41,
"memory_type": "semantic",
"domain": "work",
"importance": 0.9,
"why_retrieved": ["semantically similar (0.54)", "important", "recent"]
}
],
"total": 1
}
```
#### POST /v1/forget
```bash
curl -X POST http://localhost:7438/v1/forget \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"rid": "019d623a-..."}'
```
Response: `{"rid": "...", "found": true}`
#### POST /v1/relate
```bash
curl -X POST http://localhost:7438/v1/relate \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"entity": "Alice",
"target": "Acme",
"relationship": "works_at",
"weight": 1.0
}'
```
Response: `{"edge_id": "019d623a-..."}`
### Sessions
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/sessions` | Start a cognitive session |
| DELETE | `/v1/sessions/{id}` | End a session |
```bash
curl -X POST http://localhost:7438/v1/sessions \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"namespace": "chat-1", "client_id": "user-42"}'
```
### Cognition
| Method | Path | Description |
|--------|------|-------------|
| POST | `/v1/think` | Run consolidation + conflict scan |
| GET | `/v1/conflicts` | List open conflicts |
| POST | `/v1/conflicts/{id}/resolve` | Resolve a conflict |
| GET | `/v1/personality` | Get derived personality traits |
### Info
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/health` | Server health + cluster status (no auth required) |
| GET | `/v1/stats` | Engine statistics for the authenticated database |
| GET | `/metrics` | Prometheus-format metrics (no auth) |
#### GET /v1/health
```json
{
"status": "ok",
"engines_loaded": 1,
"cluster": {
"node_id": 1,
"role": "Leader",
"term": 5,
"leader": 1,
"accepts_writes": true,
"healthy": true
}
}
```
#### GET /metrics
Prometheus exposition format. Drops in to your existing scraping setup:
```
# HELP yantrikdb_active_memories Number of active memories
# TYPE yantrikdb_active_memories gauge
yantrikdb_active_memories{db="default"} 1247
# HELP yantrikdb_cluster_is_leader Whether this node is currently the leader
# TYPE yantrikdb_cluster_is_leader gauge
yantrikdb_cluster_is_leader{node_id="1"} 1
```
### Database management
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/databases` | List databases |
| POST | `/v1/databases` | Create a database |
### Cluster
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/cluster` | Detailed cluster status |
| POST | `/v1/cluster/promote` | Manually promote this node to leader (force election) |
#### GET /v1/cluster
```json
{
"clustered": true,
"node_id": 1,
"role": "Leader",
"current_term": 5,
"leader_id": 1,
"healthy": true,
"accepts_writes": true,
"quorum_size": 2,
"peers": [
{
"node_id": 2,
"addr": "192.168.1.2:7440",
"role": "voter",
"reachable": true,
"current_term": 5,
"last_seen_secs_ago": 0.5
},
{
"node_id": 99,
"addr": "192.168.1.3:7440",
"role": "witness",
"reachable": true,
"current_term": 5,
"last_seen_secs_ago": 0.6
}
]
}
```
## Error responses
All errors return JSON with HTTP status:
```json
{
"error": "read-only: not the leader (current leader: node 2)",
"leader": 2
}
```
| Status | Meaning |
|--------|---------|
| 200 | Success |
| 400 | Invalid request body |
| 401 | Missing or invalid Bearer token |
| 404 | Database or memory not found |
| 503 | This node is read-only (not the leader) — try the leader |
| 500 | Internal server error |
---
# Install YantrikDB Server (Linux, macOS, Windows)
URL: https://yantrikdb.com/server/install/
Install YantrikDB Server in Rust, Python, or Docker — three binaries (yantrikdb, yql, yantrikdb-witness). Pre-built releases for Linux, macOS, and Windows. Brew, Cargo, Docker, or download.
YantrikDB Server is a multi-tenant network database for cognitive memory. It exposes the YantrikDB engine over a wire protocol and HTTP gateway, with built-in embeddings, replication, automatic failover, and at-rest encryption.
## Three binaries
| Binary | Purpose | Size |
|--------|---------|------|
| `yantrikdb` | The server (data plane + cluster member) | ~22 MB |
| `yql` | Interactive REPL client (like `psql`) | ~6 MB |
| `yantrikdb-witness` | Vote-only daemon for 2-node failover | ~3 MB |
## Install via Homebrew (macOS / Linux)
```bash
brew tap yantrikos/tap
brew install yantrikdb # the server
brew install yql # interactive REPL client
brew install yantrikdb-witness # vote-only daemon for 2-node clusters
```
As of engine v0.7.0, YantrikDB ships with a **bundled default embedder** (`potion-base-2M`, ~7.9 MB, dim=64, pure Rust via `model2vec-rs`). No ONNX Runtime required for the default install — `brew install yantrikdb` is genuinely all you need. Higher-quality optional embedders (`potion-base-8M` at ~92% MiniLM and `potion-base-32M` at ~95% MiniLM) are downloadable on demand via `set_embedder_named()` from the [`yantrikos/yantrikdb-models`](https://github.com/yantrikos/yantrikdb-models) repo, SHA-256 pinned in the engine binary.
## Install via Docker
Three pre-built multi-arch images on GitHub Container Registry:
```bash
# Server (full data node)
docker pull ghcr.io/yantrikos/yantrikdb:latest
# Interactive client
docker pull ghcr.io/yantrikos/yql:latest
# Witness daemon (vote-only, 2-node cluster tiebreaker)
docker pull ghcr.io/yantrikos/yantrikdb-witness:latest
```
Run a single-node server:
```bash
docker run -d \
--name yantrikdb \
-p 7437:7437 -p 7438:7438 \
-v yantrikdb-data:/var/lib/yantrikdb \
ghcr.io/yantrikos/yantrikdb:latest
```
Run yql against it:
```bash
docker run --rm -it \
--network host \
ghcr.io/yantrikos/yql:latest \
--host localhost -p 7438 -t ydb_xxxxxxxx...
```
For a full 3-node cluster, use the [docker-compose.yml](https://github.com/yantrikos/yantrikdb-server/blob/main/docker/docker-compose.yml) example in the repo.
## Install via Cargo
```bash
cargo install yantrikdb-server
cargo install yql
cargo install yantrikdb-witness
```
This builds from source against your local glibc — useful on older Linux distros where the pre-built binaries don't run. Requires the Rust toolchain.
## Download pre-built binaries
```bash
# Linux
wget https://github.com/yantrikos/yantrikdb-server/releases/latest/download/yantrikdb-linux-amd64
wget https://github.com/yantrikos/yantrikdb-server/releases/latest/download/yql-linux-amd64
chmod +x yantrikdb-linux-amd64 yql-linux-amd64
sudo mv yantrikdb-linux-amd64 /usr/local/bin/yantrikdb
sudo mv yql-linux-amd64 /usr/local/bin/yql
```
Other platforms available on the [releases page](https://github.com/yantrikos/yantrikdb-server/releases/latest):
- `yantrikdb-windows-amd64.exe`
- `yantrikdb-macos-arm64` (Apple Silicon)
- `yantrikdb-macos-amd64` (Intel)
Each binary is also published for `yql` and `yantrikdb-witness`.
## Which method should I use?
| Method | Best for |
|--------|----------|
| **Homebrew** | Mac developers — fastest |
| **Docker** | Reproducible deployments — isolated, multi-arch, easy to upgrade |
| **Cargo** | Older Linux distros, contributors, anyone with Rust installed |
| **Pre-built binary** | Servers with no package manager / no Rust toolchain |
## Optional: ONNX Runtime for fastembed-based embedders
**You don't need ONNX for the default install.** As of engine v0.7.0, YantrikDB ships with a bundled `potion-base-2M` static embedder (pure Rust, no ONNX, ~7.9 MB, dim=64). It works out of the box for most agent-memory workloads.
ONNX Runtime is only needed when you opt into a `fastembed`-based embedder for higher recall quality (typically `BGEBaseENV15`, dim=384). For that, you need ORT 1.23+ available at runtime:
### Linux
```bash
# Debian 13 / Ubuntu 24.04+
sudo apt-get install libonnxruntime1.21
# Or download from Microsoft directly:
wget https://github.com/microsoft/onnxruntime/releases/download/v1.24.4/onnxruntime-linux-x64-1.24.4.tgz
tar xzf onnxruntime-linux-x64-1.24.4.tgz
sudo cp onnxruntime-linux-x64-1.24.4/lib/libonnxruntime*.so* /usr/local/lib/
sudo ldconfig
export ORT_DYLIB_PATH=/usr/local/lib/libonnxruntime.so.1.24.4
```
### macOS
```bash
brew install onnxruntime
export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib
```
### Windows
Install ONNX Runtime via [Microsoft's release page](https://github.com/microsoft/onnxruntime/releases) and set `ORT_DYLIB_PATH` to `onnxruntime.dll`.
### Skip embeddings (client-only mode)
If you don't want server-side embedding (clients send pre-computed vectors):
```toml
# yantrikdb.toml
[embedding]
strategy = "client_only"
```
## Glibc requirement
Linux binaries are built on Ubuntu (glibc 2.39). They run on:
- ✅ Debian 13 (trixie) and newer
- ✅ Ubuntu 22.04 and newer
- ✅ Fedora 38 and newer
- ❌ Debian 12 (bookworm) — glibc 2.36, too old
If you're on an older distro, install from `cargo install yantrikdb-server` which builds against your local glibc.
## Verify
```bash
yantrikdb --help
yql --help
yantrikdb-witness --help
```
## Next steps
- [Quick Start](/server/quickstart/) — single-node setup in 60 seconds
- [Cluster Deployment](/server/cluster/) — replication + auto-failover
- [yql REPL](/server/yql/) — interactive client
---
# YantrikDB Server Quick Start — Single-Node in 60 Seconds
URL: https://yantrikdb.com/server/quickstart/
Get a YantrikDB server running on Linux, macOS, or Windows in under 60 seconds — install, start, store your first memory, recall by semantic search. Single-node setup for AI agent memory.
## Single-node mode
Spin up a YantrikDB server on your machine, create a database, mint a token, and start storing memories.
### 1. Create the database and a token
```bash
yantrikdb db --data-dir ./data create default
yantrikdb token --data-dir ./data create --db default
# Prints: ydb_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
Save that token.
### 2. Start the server
```bash
yantrikdb serve --data-dir ./data
```
The server is now listening on:
- **Wire protocol**: `0.0.0.0:7437` (binary, multiplexed, fast)
- **HTTP gateway**: `0.0.0.0:7438` (REST + JSON)
### 3. Talk to it from yql
In another terminal:
```bash
yql --host localhost -t ydb_xxxxxxxx...
```
```
yql connected to http://localhost:7438
type \h for help, \q to exit
yantrikdb> remember "Alice leads engineering at Acme" importance=0.9 domain=work
✓ stored: 019d623a-3d70-712e-9315-e1da5ee41114
yantrikdb> recall who leads engineering top=5
+---+-------+---------------------------------+--------+
| # | score | text | domain |
+---+-------+---------------------------------+--------+
| 1 | 1.41 | Alice leads engineering at Acme | work |
+---+-------+---------------------------------+--------+
yantrikdb> relate Alice -> Acme as works_at
✓ edge: 019d623a-41cf-71a2 (Alice -[works_at]-> Acme)
yantrikdb> \stats
yantrikdb> \q
```
### 4. Or talk to it via HTTP
```bash
TOKEN=ydb_xxxxxxxx...
curl -X POST http://localhost:7438/v1/remember \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"text": "First memory", "importance": 0.9, "domain": "work"}'
curl -X POST http://localhost:7438/v1/recall \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"query": "first memory", "top_k": 5}'
```
### 5. Or talk to it from Python
```bash
pip install yantrikdb-client
```
```python
from yantrikdb import connect
db = connect("http://localhost:7438", token="ydb_xxxxxxxx...")
db.remember("Alice leads engineering", importance=0.9, domain="work")
results = db.recall("who leads engineering?", top_k=5)
for r in results.results:
print(f"[{r.score:.2f}] {r.text}")
```
## What's running
A single `yantrikdb serve` process gives you:
- **Multi-tenant database engine** — create as many databases as you need with `yantrikdb db create `
- **Built-in embeddings** — all-MiniLM-L6-v2 via fastembed (no API key)
- **Background workers** — autonomous consolidation, decay, conflict detection
- **HTTP + wire protocol** — bring any client
- **Bearer token auth** — per-database access control
- **Prometheus metrics** — `GET /metrics`
## Maturity
**Read this before betting your project on it.**
YantrikDB Server is **v0.8.9 — substrate-batch alpha**. The v0.8 line shipped the full RFC 010 commit substrate (mutation commit log + openraft consensus + Jepsen debug history), RFC 014-A cluster mTLS, RFC 011-A forget interface, RFC 009 admission control, RFC 013-A HNSW manifest, and RFC 019 durable jobs/leases. v0.8.9 added a read-connection pool that eliminates recall serialisation (single-AGI clients no longer clog one CPU core). Running live on a 3-node Proxmox cluster with multiple tenants.
### What's stable
- ✅ The embeddable engine (`yantrikdb` crate / `pip install yantrikdb`) — used in production since early 2026
- ✅ The wire protocol and HTTP gateway — 45+ tests including integration, compatibility, and crash recovery
- ✅ openraft consensus (replaces raft-lite from v0.5.x) — proper leader election, log replication, snapshots
- ✅ Single-node mode — no replication overhead, drop-in for any HTTP-speaking client
- ✅ At-rest encryption (AES-256-GCM) — uses the engine's audited crypto path
- ✅ Cluster mTLS (RFC 014-A) — encrypted + mutually-authenticated cluster transport
- ✅ Admission control (RFC 009) — per-tenant quotas + circuit breaker, fail-degraded-conservative
- ✅ Mutation commit log (RFC 010-A) — total-ordered, content-addressed substrate behind every write
- ✅ Read-connection pool (v0.8.9) — concurrent recall scales linearly with cores instead of serialising
- ✅ Eager engine warm-up at startup (v0.8.8) — first recall hits a loaded engine, no 8-second cold-load timeout
- ✅ `parking_lot` mutexes everywhere — runtime deadlock detection (10s cadence)
- ✅ Per-handler Prometheus metrics (latency histograms, lock-hold histograms, request counters)
### Performance (v0.8.9, Docker on Windows host, 1 vCPU)
| Workload | Result |
|---|---|
| 10 concurrent recall | 10/10 succeed, p99 143 ms |
| 50 concurrent recall | 33×200, 17×503 admission shed, 0 timeouts |
| Forget invariant under race | 0 stale reads (5 readers × 5 probes) |
### What's not done yet
- ❌ Performance benchmarks at scale (10M+ memories)
- ❌ LongMemEval head-to-head vs Zep / Memento / Mastra (tracked, M5)
- ❌ Encrypted-cluster embedding backfill on followers (engine method `encrypt_embedding` is `pub(crate)`; workaround: don't enable encryption + replication together)
- ❌ Reference deployments outside the maintainer's homelab
### When to use it
| Use case | Recommendation |
|----------|---------------|
| Personal agent memory, single user | ✅ Go for it |
| Self-hosted homelab cluster | ✅ Go for it — hardened and chaos-tested |
| Backend for an open-source AI tool | ✅ Single-node mode is solid |
| Production SaaS with thousands of users | ⏸ Wait for v1.0, or be the lead user |
| Mission-critical, customer-facing data | ⏸ Not yet — pin to a release, run it in parallel with your existing stack first |
We'd rather you trust us in the long run than deploy it now and get burned. If you're an early adopter looking for a real cognitive memory database to grow with — welcome. File issues, file PRs, run it on your homelab, tell us what breaks.
## Next steps
- [Cluster Deployment](/server/cluster/) — add replication and auto-failover
- [yql REPL](/server/yql/) — full command reference
- [HTTP API](/server/http-api/) — REST endpoint reference
- [Encryption](/server/encryption/) — at-rest AES-256-GCM
---
# YantrikDB Wire Protocol Specification — Binary Frame Format
URL: https://yantrikdb.com/server/wire-protocol/
The binary wire protocol used by YantrikDB clients and peer-to-peer cluster transport. Length-prefixed frames, versioned request/response, mTLS-wrapped. Spec for SDK authors and operators.
YantrikDB Server speaks a custom binary wire protocol on port `7437` alongside the JSON HTTP gateway on port `7438`. The wire protocol is faster (multiplexed, binary-encoded), supports streaming responses, and is the same protocol nodes use to talk to each other.
The codec is published as the [`yantrikdb-protocol`](https://crates.io/crates/yantrikdb-protocol) crate.
## Frame format
Every message is a length-prefixed frame:
```
┌──────────┬──────────┬──────────┬──────────┬────────────┐
│ Length │ Version │ OpCode │ StreamID │ Payload │
│ (4 bytes)│ (1 byte) │ (1 byte) │ (4 bytes)│ (variable) │
└──────────┴──────────┴──────────┴──────────┴────────────┘
```
- **Length**: big-endian u32, total bytes after this field
- **Version**: protocol version + flag bits
- Bit 7 (0x80): JSON payload mode (debug)
- Bit 6 (0x40): zstd-compressed payload
- **OpCode**: u8, identifies the message type (see table below)
- **StreamID**: big-endian u32, allows multiplexing concurrent requests on one connection
- **Payload**: MessagePack-serialized message struct, optionally zstd-compressed
## Connection lifecycle
```
Client Server
│ │
├── AUTH(token="ydb_abc123") ──────────► │
│ ├── validate token
│ ◄──────────────── AUTH_OK(db="default")│
│ │
├── REMEMBER(text="...") ──────────────► │
│ ◄──────── REMEMBER_OK(rid="mem_01") │
│ │
├── RECALL(query="...") ────────────────► │
│ ◄──────── RECALL_RESULT(mem_01, 0.93) │
│ ◄──────── RECALL_RESULT(mem_07, 0.71) │
│ ◄──────── RECALL_END(total=2) │
│ │
```
The recall response is **streamed** — each result is its own frame, ended by a `RECALL_END` frame. This lets clients start processing results before the server has finished computing them all.
## OpCode table
### Auth (0x01–0x03)
| Code | Name | Direction |
|------|------|-----------|
| 0x01 | `AUTH` | C→S |
| 0x02 | `AUTH_OK` | S→C |
| 0x03 | `AUTH_FAIL` | S→C |
### Memory (0x20–0x32)
| Code | Name | Direction |
|------|------|-----------|
| 0x20 | `REMEMBER` | C→S |
| 0x21 | `REMEMBER_OK` | S→C |
| 0x22 | `REMEMBER_BATCH` | C→S |
| 0x30 | `RECALL` | C→S |
| 0x31 | `RECALL_RESULT` | S→C (streamed) |
| 0x32 | `RECALL_END` | S→C |
### Graph (0x40–0x43)
| Code | Name |
|------|------|
| 0x40 | `RELATE` |
| 0x41 | `RELATE_OK` |
| 0x42 | `EDGES` |
| 0x43 | `EDGES_RESULT` |
### Lifecycle (0x50–0x71)
| Code | Name |
|------|------|
| 0x50 | `FORGET` |
| 0x51 | `FORGET_OK` |
| 0x60 | `SESSION_START` |
| 0x61 | `SESSION_END` |
| 0x62 | `SESSION_OK` |
| 0x70 | `THINK` |
| 0x71 | `THINK_RESULT` |
### Cluster / Replication (0xC0–0xCF)
| Code | Name | Direction |
|------|------|-----------|
| 0xC0 | `CLUSTER_HELLO` | peer→peer |
| 0xC1 | `CLUSTER_HELLO_OK` | peer→peer |
| 0xC2 | `OPLOG_PULL` | peer→peer |
| 0xC3 | `OPLOG_PULL_RESULT` | peer→peer |
| 0xC4 | `OPLOG_PUSH` | peer→peer |
| 0xC5 | `OPLOG_PUSH_OK` | peer→peer |
| 0xC6 | `HEARTBEAT` | leader→follower |
| 0xC7 | `HEARTBEAT_ACK` | follower→leader |
| 0xC8 | `REQUEST_VOTE` | candidate→voter |
| 0xC9 | `VOTE_GRANTED` | voter→candidate |
| 0xCA | `VOTE_DENIED` | voter→candidate |
| 0xCB | `CLUSTER_STATUS` | C→S |
| 0xCC | `CLUSTER_STATUS_RESULT` | S→C |
| 0xCD | `READONLY_ERROR` | S→C |
| 0xCE | `CLUSTER_DATABASE_LIST` | peer→peer |
| 0xCF | `CLUSTER_DATABASE_LIST_RESULT` | peer→peer |
### Control (0xF0–0xF2)
| Code | Name |
|------|------|
| 0xF0 | `ERROR` |
| 0xF1 | `PING` |
| 0xF2 | `PONG` |
## Compression
Large payloads (oplog batches, recall results) are auto-compressed with zstd when they exceed 4KB. The compression flag (bit 6 of the version byte) tells the receiver to decompress before unpacking the MessagePack body.
Compressed payloads typically reach **3–5× compression ratio** for natural language memory text.
## Implementing a client
You have three options:
1. **Use the Rust crate**: `cargo add yantrikdb-protocol` — gives you the full Frame/codec/message types
2. **Use the HTTP gateway**: simpler, just JSON over HTTPS — most clients should do this
3. **Implement from scratch**: follow the frame format above. The MessagePack message structs are documented in the protocol crate's [source code](https://github.com/yantrikos/yantrikdb-server/blob/main/crates/yantrikdb-protocol/src/messages.rs)
## Why a custom protocol?
YantrikDB workloads are chatty (5–20 ops per agent turn), session-aware, and benefit from streaming. HTTP works fine for occasional clients but adds overhead per request. The wire protocol gives:
- **Lower latency** — single connection, no per-request HTTP parsing
- **Multiplexing** — multiple streams on one connection
- **Streaming recall** — process results as they arrive
- **Server push** — events (conflicts, decay, triggers) flow without polling
- **Native binary** — no JSON parse on the hot path
The HTTP gateway is the universal interface; the wire protocol is the optimized one.
---
# yql — YantrikDB Interactive REPL Client (like psql)
URL: https://yantrikdb.com/server/yql/
yql is the interactive REPL for YantrikDB Server — query cognitive memory, manage databases, inspect cluster state from the command line. The psql equivalent for AI agent memory. Auth via Bearer token, JSON output, scriptable.
`yql` is the interactive client for YantrikDB Server. Think `psql` for cognitive memory: connect, run commands, get pretty tables back.
It's a separate binary, MIT-licensed (the server is AGPL).
## Install
```bash
cargo install yql
```
Or download a pre-built binary:
```bash
wget https://github.com/yantrikos/yantrikdb-server/releases/latest/download/yql-linux-amd64
chmod +x yql-linux-amd64
mv yql-linux-amd64 /usr/local/bin/yql
```
Available for `linux-amd64`, `windows-amd64`, `macos-arm64`, `macos-amd64`.
## Connect
```bash
yql --host localhost -p 7438 -t ydb_your_token_here
```
Or set the token via environment variable:
```bash
export YQL_TOKEN=ydb_xxxxxxxx...
yql --host localhost
```
## Usage
```
yql connected to http://localhost:7438
type \h for help, \q to exit
yantrikdb> remember "Alice leads engineering at Acme" importance=0.9 domain=work
✓ stored: 019d623a-3d70-712e-9315-e1da5ee41114
yantrikdb> recall who leads engineering top=5
+---+-------+---------------------------------+--------+--------------------------------+
| # | score | text | domain | why |
+---+-------+---------------------------------+--------+--------------------------------+
| 1 | 1.41 | Alice leads engineering at Acme | work | semantically similar (0.54)... |
+---+-------+---------------------------------+--------+--------------------------------+
(1 rows)
yantrikdb> relate Alice -> Acme as works_at
✓ edge: 019d623a-41cf-71a2 (Alice -[works_at]-> Acme)
yantrikdb> \cluster
yantrikdb> \stats
yantrikdb> \q
```
## Memory commands (natural language)
| Command | Purpose |
|---------|---------|
| `remember "" [importance=0.9] [domain=work]` | Store a memory |
| `recall [top=10] [domain=work]` | Semantic search |
| `forget ` | Tombstone a memory |
| `relate -> as ` | Create graph edge |
### Optional fields for `remember`
- `importance=0.9` (0.0-1.0)
- `domain=work`
- `source=user`
- `memory_type=semantic` (or `episodic`, `procedural`)
- `valence=0.5` (-1.0 to 1.0, emotional tone)
- `half_life=168` (decay half-life in hours)
- `certainty=0.95`
### Filters for `recall`
- `top=10` — number of results
- `domain=work` — limit to a domain
- `source=user` — limit to a source
- `memory_type=semantic`
## Meta commands (psql-style)
| Command | Alias | Purpose |
|---------|-------|---------|
| `\stats` | `\s` | Engine statistics |
| `\dt` | `\l` | List databases |
| `\conflicts` | `\c` | List open conflicts |
| `\personality` | `\p` | Derived personality traits |
| `\think` | `\t` | Run consolidation + conflict scan |
| `\cluster` | | Cluster status (replication / failover) |
| `\health` | | Server health check |
| `\json ` | | Raw GET request to any endpoint |
| `\help` | `\h` `\?` | Show help |
| `\quit` | `\q` | Exit |
## Non-interactive mode
Run a single command and exit:
```bash
yql --host localhost -t $TOKEN -c '\stats'
yql --host localhost -t $TOKEN -c 'remember "hello" importance=0.8'
yql --host localhost -t $TOKEN -c 'recall hello'
```
Useful for shell scripts, cron jobs, monitoring.
## Command history
`yql` saves your command history to `~/.yql_history` so you can up-arrow through previous commands across sessions. Same as `psql`.
---
# YantrikDB Showcase — Real Cognitive Memory Use Cases
URL: https://yantrikdb.com/showcase/
Eight real experiments demonstrating YantrikDB's cognitive memory architecture — polarity contradictions, validity windows, source attribution, and entity-graph reconstruction across journalism, legal, financial forensics, multi-agent ops, and historical research.
Eight showcases. Eight domains. One primitive: structured claims with polarity, validity, and source attribution — and automatic contradiction detection across them.
**Each showcase answers the same question: "why couldn't Postgres + embeddings + a dashboard do this?"**
---
## Multi-Agent AI
### [Multi-Agent Ops: Which Agent Memory Is Stale?](/showcase/multi-agent/)
> Five agents watched the same Black Friday incident. The database figured out which memory was stale.
Five sub-agents report from five different sources. Two are stale. YantrikDB preserves every claim with validity windows and confidence bands, then surfaces which beliefs are live and which are holding yesterday's truth.
**Belief management under contradiction. Not a vector store — a witness stand.**
---
## Public Claims vs Records
### [Volkswagen Dieselgate](/showcase/volkswagen/)
> The car passed emissions — because it knew it was being tested.
Twelve years of public record: VW compliance claims, internal engineering documents, ICCT field tests, EPA Notice of Violation, DOJ consent decree. Five polarity contradictions on one tuple. Validity windows span 2006 to 2017.
---
## Legal Discovery
### [Testimony vs Machine Logs](/showcase/legal-discovery/)
> He said he never touched the repo. Badge, VPN, and git all say he did.
A fictional trade-secrets matter (Waymo v. Uber-inspired patterns). Sworn deposition claims coexist with six forensic sources. The evidence chain is the query result.
---
## Financial Forensics
### [Wirecard — The €1.9B That Existed and Didn't](/showcase/wirecard/)
> The €1.9B both existed and didn't — depending on which source you asked.
The same number, reported across four sources, took four contradictory positions over six years. Eight polarity contradictions on one tuple. The temporal query flips the belief state between 2019 and 2020.
---
## Investigative Journalism
### [Follow the Money Through the Entity Graph](/showcase/journalism/)
> The campaign denied taking the money. The filings, shell-company records, and ownership chain traced it anyway.
A fictional five-hop entity chain across FEC filings, Delaware registry, bank transfers, and industry classification. The contradiction lives not in any single source but in the composition across five.
---
## Incident Response
### [The Rashomon Engine](/showcase/rashomon/)
> David said he didn't touch the repo. Git says he did, 23 minutes before the leak.
Five witnesses to a data breach, plus badge and git logs. Structured claims with polarity let the engine identify the perpetrator and cite the exact lies — with real queries, not scripted narrative.
---
## Historical Research
### [Watergate: What the Tapes Caught](/showcase/watergate/)
> Nixon said nobody in the White House was involved. The tape of his own voice says otherwise, recorded six days after the break-in.
Fifty years of declassified primary sources: Nixon's public denials, the White House tapes, sworn Senate testimony, Senate Watergate Committee findings. Six polarity contradictions on Nixon alone, in one query.
---
## Character-Informed Generation
### [Shakespeare: Bringing a Character Alive](/showcase/shakespeare/)
> 207 memories. 288 entities. Personality derived. Then he wrote.
The difference between 29 memories (generic pastiche) and 207 (a specific man writing to a specific woman at a specific hour). Richer memory → richer character → richer output from any LLM.
---
*Want to add your own showcase? Open a [discussion](https://github.com/yantrikos/yantrikdb-server/discussions) or submit a PR to the [website repo](https://github.com/yantrikos/yantrikdb-web).*
---
# Investigative Journalism: Follow the Money Through the Entity Graph
URL: https://yantrikdb.com/showcase/journalism/
The candidate denied receiving pharma money. The filings, shell-company records, and ownership chain traced it anyway — five hops through public registries. YantrikDB treats the contradiction between a direct denial and a derived entity chain as the same kind of query.
> **The campaign denied taking the money. The filings, shell-company records, and ownership chain traced it anyway.**
A fictional campaign-finance investigation inspired by documented patterns from closed real cases (Abramoff-era lobbying networks, FEC-disclosed shell-entity donor pathways, and dozens of ProPublica/OpenSecrets reconstructions). All names are invented. The reconstruction pattern is how real journalism works.
This is the **investigative journalism** showcase — demonstrating that YantrikDB surfaces contradictions not just from direct statement-vs-statement mismatches, but from **entity resolution across a graph of ownership and funding**.
---
## The setup
**Marcus Lanier**, a fictional U.S. Senate candidate, is campaigning statewide. At a July 2026 town hall, asked about pharmaceutical campaign funding, he says:
> *"I have never taken a dollar from anyone connected to the pharmaceutical industry. Never have, never will."*
A reporter spends three months pulling records from public registries. The story: technically, the statement is defensible at every single hop. Collectively, it's false.
### Sources
| Source | Authority | What it contains |
|---|---|---|
| `public.lanier` | Direct statement | The denial |
| `fec.filings` | Federal Election Commission | Itemized contributions to campaign |
| `fec.pac_disclosures` | FEC | The PACs' own funding sources |
| `delaware.registry` | Delaware Div. of Corporations | LLC ownership chains |
| `propublica.profile` | NAICS + SEC EDGAR | Industry classification |
| `reporter.bank_records` | Source-reporter corroboration | Wire transfer records |
---
## The entity chain the engine reconstructed
Starting from `Lanier_campaign`, walking backwards through the claims ledger:
```
Hop 1 — direct donors:
Better_Tomorrow_Action_Fund -> Lanier_campaign [fec.filings]
Progressive_Health_Futures_PAC -> Lanier_campaign [fec.filings]
Hop 2 — who funded the donor PACs:
Meridian_Public_Affairs_Group -> Better_Tomorrow... [fec.pac_disclosures]
Windhaven_Strategies_LLC -> Progressive_Health... [fec.pac_disclosures]
Hop 3 — who owns the funding LLCs:
Carrington_Horizon_Holdings owns Meridian_Public_Affairs_Group [delaware.registry]
Carrington_Horizon_Holdings owns Windhaven_Strategies_LLC [delaware.registry]
Hop 4 — ultimate beneficial owner:
Kellner_Therapeutics_Group owns Carrington_Horizon_Holdings [delaware.registry]
Hop 5 — industry classification:
Kellner_Therapeutics_Group is a member of pharma_industry [propublica.profile]
```
Five hops across five independent public registries. Every hop is a structured claim with source attribution. YantrikDB walks the chain in one query.
---
## The contradiction
```
Lanier_campaign --received_funds_from--> pharma_industry
(public.lanier) CLAIMS NO
(derived from entity chain) CLAIMS YES
```
Lanier's statement is *not* contradicted by a single opposing assertion — no single source says *"Lanier took pharma money."* The contradiction emerges from the **composition** of five claims across five sources. That's what makes it an investigative-journalism showcase: the structured lie is only visible if your memory system can walk the entity graph and reason over ownership relations.
---
## The temporal query shows belief changing with the reporting
**Before the town hall (2026-04-01):** the public record shows only `public.lanier`'s denial (which doesn't exist yet) and the fragments of donations that haven't been connected. No single source contradicts him. The lie is defensible.
**After the reporting (2026-08-01):** the entity chain is assembled. The denial and the chain coexist in the ledger. The composition is the story.
Same database. Same claims. The journalistic question *"what did the public record show on date X?"* has a different answer at different points — because validity windows and source attribution are first-class.
---
## Why couldn't Postgres + embeddings + a dashboard do this?
A keyword search or vector database would not find this story. None of the source documents mention each other textually. The Delaware LLC filing does not name Lanier. The FEC Form 3 doesn't name Kellner Therapeutics. The town hall quote doesn't name any PAC. The connection is **ownership**, not linguistic similarity.
A graph database could model the ownership chain — but it has no notion of polarity or validity. It cannot represent Lanier's *"I never took pharma money"* (polarity=-1) as a claim that contradicts the **derived** polarity=+1 fact that follows from the ownership graph.
YantrikDB stores the direct denial and every hop of the entity chain as structured claims with source attribution and polarity. Walking the chain is one query per hop. Surfacing the contradiction at the top is the query result.
That's the category.
---
## What this unlocks
Any investigation where the contradiction lives in the composition of public records:
- **Campaign finance** — the scenario above
- **Sanctions & beneficial ownership** — stated non-exposure vs shell-entity chain evidence
- **Supply-chain traceability** — vendor certifications vs audit-trail contradictions via subcontractors
- **Money-laundering investigations** — ledger entries that individually pass but fail together
- **Medical ethics** — stated independence vs cross-referenced affiliations via foundations and consulting
- **Regulatory capture** — public lobbying disclosures vs the entity network they connect
Every one of these is an entity-graph problem with contradictions hiding at composition layers. Keyword and vector search miss them. YantrikDB surfaces them.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/journalism_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [journalism_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/journalism_engine.py)
---
*The denial and the chain, in one database. Follow the money.*
---
# Legal Discovery: Testimony vs Machine Logs
URL: https://yantrikdb.com/showcase/legal-discovery/
A sworn deposition says she never touched the repo. Badge, VPN, git, USB, and an email trail all say she did. YantrikDB surfaces the exact contradictions that decide discovery — in one query.
> **He said he never touched the repo. Badge, VPN, and git all say he did.**
A fictional trade-secrets matter inspired by patterns documented in public trade-secret litigation (Waymo v. Uber and others). All names and events are invented. The reconstruction pattern is real.
This is the **legal discovery** showcase — demonstrating how YantrikDB treats sworn testimony and machine evidence as coexisting claims, with automatic contradiction detection at the (subject, relation, object) level.
---
## The matter
**Summit Atlas, Inc. v. Polaris Robotics.** Summit alleges that Priya Ramanathan, a former senior engineer, downloaded proprietary LIDAR firmware before joining competitor Polaris.
At deposition (2026-08-15), Ramanathan denies everything. The forensic record tells a different story.
### Sources
| Source | Authority | Content |
|---|---|---|
| `deposition.ramanathan` | Sworn testimony | Direct denials under oath |
| `system.badge` | Kastle access logs | Building/floor entry timestamps |
| `system.vpn` | Corporate VPN logs | Remote session records |
| `system.git` | GitLab server logs | Clone/download events per user |
| `system.dlp` | Endpoint DLP (CrowdStrike) | USB attach/write audit |
| `system.email` | Preserved email archive | Pre-departure correspondence |
---
## What the engine produced
### Phase 3: Polarity contradictions
```
[1] Ramanathan --accessed--> lidar_firmware_repo
(deposition.ramanathan) CLAIMS NO
(system.git) CLAIMS YES at 2026-05-18 23:02
(system.git) CLAIMS YES at 2026-05-24 22:08
[2] Ramanathan --copied_to--> removable_media
(deposition.ramanathan) CLAIMS NO
(system.dlp) CLAIMS YES at 2026-05-24 22:47
```
Two sworn denials, two polarity contradictions, four forensic sources backing the opposite. Each side coexists in the claims ledger with its own provenance.
### Phase 4: Temporal query — "what did discovery know on 2026-05-25?"
```
[system.dlp] YES Ramanathan --copied_to--> removable_media (22:47–22:51)
[system.git] YES Ramanathan --accessed--> lidar_firmware_repo (22:08–22:10)
[system.vpn] YES Ramanathan --accessed--> SummitAtlas_network (22:41–23:55)
[system.badge] YES Ramanathan --was_at--> SummitAtlas_R&D_wing (20:47–23:12)
[deposition.ramanathan] NO Ramanathan --copied_to--> removable_media
[deposition.ramanathan] NO Ramanathan --accessed--> lidar_firmware_repo
```
Every machine source agrees; the sworn denial from three months later sits alongside them. Discovery counsel can literally query the database for *"the state of the factual record as of May 25"* — and get a structured answer with full provenance.
### Phase 5: The recall chain pins the "smoking email"
Within the top 8 recall results, the database surfaces:
```
[system.dlp] 2026-05-24 22:47 Samsung T7 SSD attached, 2.8 GB written
[deposition.ramanathan] "Absolutely not. That would have violated my NDA."
[system.git] 2026-05-18 23:02 cloned lidar-firmware/titan-v3 (2.4 GB)
[system.email] 2026-05-02 "I'll have a small package ready to bring over"
[system.git] 2026-05-24 22:08 downloaded ZIP snapshot
[system.badge] entered R&D wing 20:47
[system.badge] exited R&D wing 23:12
```
The denial, the clone, the copy, the badge session, and the pre-departure email to the competitor's recruiter — all ranked together because they all matter to the same question.
---
## Why couldn't Postgres + embeddings + a dashboard do this?
Most legal-tech tools do retrieval or timeline generation. They find documents that match keywords, or order events by timestamp. None of them treat *"Ramanathan says she didn't access the repo"* and *"the git server says she did"* as **two coexisting structured claims on the same `(subject, relation, object)` tuple**, with opposite polarity, source attribution, validity windows, and automatic contradiction detection.
A SQL database would force one value to overwrite the other. A vector database would return both as "similar" with no notion that they contradict. A graph database could model the people and events but has no polarity on its edges — it can't distinguish *"A claims X"* from *"X is true"*.
That's what YantrikDB does. That's the category.
---
## What this unlocks
The pattern generalizes to every matter where sworn statements must be reconciled against documentary and machine evidence:
- **Trade secrets / IP theft** — the scenario above
- **Employment disputes** — testimony vs HR logs, Slack, email
- **Financial fraud** — depositions vs transaction records
- **Antitrust** — executive testimony vs internal communications
- **Regulatory enforcement** — sworn filings vs operational data
- **Whistleblower cases** — company statements vs internal records
Every one of these becomes the same kind of structured contradiction reconstruction. The evidence chain is the query result.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/legal_discovery_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [legal_discovery_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/legal_discovery_engine.py)
---
*Sworn testimony and machine logs, in the same database, as coexisting contradictory claims. That's what discovery actually needs.*
---
# Multi-Agent Ops: Which Agent Memory Is Stale?
URL: https://yantrikdb.com/showcase/multi-agent/
Five AI sub-agents watched the same Black Friday incident. They disagreed. The database figured out which memory was stale — with validity windows, source attribution, and polarity contradictions.
> **Five agents watched the same incident. The database figured out which memory was stale.**
This is the **multi-agent AI** showcase — the one that matters most to anyone building agent fleets today.
A normal agent stack collapses disagreement into one averaged answer. YantrikDB preserves every claim with its source, validity window, and confidence band — then tells the coordinator exactly which beliefs are fresh, which are stale, and which contradict each other.
---
## The scenario
**Black Friday 2026, 15:20 UTC.** The on-call engineer at a large e-commerce platform opens the incident coordinator and asks:
> *"Is the checkout rollout active? Are customers impacted?"*
Five sub-agents have been watching five different sources. They all believe they're telling the truth. Two of them are working from stale data.
### The five agents
| Agent | Source | State at 15:20 |
|---|---|---|
| `agent.deploy` | CI/CD pipeline | ✅ Fresh — rollout completed 15:10 |
| `agent.telemetry` | Prometheus metrics | ✅ Fresh — error spike at 15:13 |
| `agent.support` | Customer support inbox | ✅ Fresh — 3 tickets since 15:12 |
| `agent.config` | Feature-flag API snapshot | ❌ **Stale** — last polled at 14:50 |
| `agent.status` | Public status page scrape | ❌ **Stale** — last updated 14:40 |
Each agent writes structured claims with `extractor`, `valid_from`, `valid_to`, and `confidence_band`. None of them gets to flatten the truth.
---
## What the engine produces
### Phase 3: Polarity contradictions
```
[1] POLARITY_CONTRADICTION
checkout_rollout --is_active--> true
(agent.deploy) CLAIMS YES [15:10 - now] conf=high
(agent.config) CLAIMS NO [14:50 - 15:10] conf=high
[2] POLARITY_CONTRADICTION
checkout_service --is_healthy--> true
(agent.status) CLAIMS YES [14:40 - 15:13] conf=medium
(agent.telemetry) CLAIMS NO [15:13 - now] conf=high
```
Both contradictions are real — at the moment each claim was written, each agent was correct. The validity windows are what disambiguate.
### Phase 4: The temporal query
**"What did we believe at 15:20?"**
```
checkout_rollout YES [agent.deploy, 15:10–now, high]
checkout_service NO [agent.telemetry, 15:13–now, high]
customer_impact YES [agent.support, 15:12–now, medium]
```
**"What would we have believed at 14:55?"**
```
checkout_rollout NO [agent.config, 14:50–15:10, high]
checkout_service YES [agent.status, 14:40–15:13, medium]
```
Same database. Same claims. Different moment in time = different truth. The on-call engineer can query the fleet's belief state at any point in history, because validity windows are first-class.
### Phase 6: The reconciled answer
```
Current facts (chosen by source-freshness + confidence):
checkout_rollout --is_active--> true = YES [authority: agent.deploy]
checkout_service --is_healthy--> true = NO [authority: agent.telemetry]
Agents with STALE beliefs (excluded from verdict):
- agent.config
- agent.status
Recommended action for the on-call engineer:
* checkout_v8 rollout IS live (deploy completed at 15:10)
* checkout service IS degraded (telemetry confirms error spike)
* customer impact IS real (support tickets arriving)
* ROLL BACK checkout_v8 via feature flag
```
---
## Why couldn't Postgres + embeddings + a dashboard do this?
A vector database returns all 5 agent observations as "similar" and lets the LLM guess which to trust. A SQL database stores each agent's state as flat rows — but the moment `agent.config` is updated, the old value is overwritten and the 14:50 snapshot is gone. A graph database can model relationships but has no notion of polarity or validity.
None of them can store `agent.deploy`'s *YES* and `agent.config`'s *NO* on the same `(checkout_rollout, is_active, true)` tuple as two coexisting rows with opposite polarity and non-overlapping validity windows, automatically flag them as a contradiction, and let you query *"what did we believe at 14:55 vs 15:20?"* — all in one round-trip.
That's what YantrikDB does. That's the category.
---
## What this unlocks
Every agent fleet today has this problem:
- **RAG/retrieval agents** — different retrievers surface different documents; current stacks average them, losing the disagreement
- **Tool-using agents** — the API says one thing, the cached result says another; current stacks pick a random winner
- **Monitoring/ops agents** — five dashboards, five opinions; current stacks rely on the human to reconcile
- **Multi-modal agents** — vision says X, audio says Y, text says Z; current stacks force early commitment
With YantrikDB, every agent's observation becomes a **claim with provenance, validity, and polarity**. Contradiction detection fires automatically. The coordinator asks not *"what's the answer?"* but *"what beliefs are live right now, and which sources have gone stale?"*
This is not agent orchestration. This is **belief management under contradiction** — and no other memory system ships it.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/multi_agent_ops_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [multi_agent_ops_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/multi_agent_ops_engine.py)
---
*Your agents disagreed. The database knew which memory was stale.*
---
# The Rashomon Engine: Truth from Conflicting Testimony
URL: https://yantrikdb.com/showcase/rashomon/
Five witnesses, two log sources, some of them lying. Watch YantrikDB reconstruct what actually happened — with real queries, not scripted narrative.
**What happens when you give YantrikDB five witness statements about a data breach — some of them deliberately lying — plus badge logs and git logs as ground truth?**
The engine names the perpetrator, cites the exact lies, and explains its reasoning. The final verdict is computed from the claims ledger, not scripted in Python.
No other memory system can do this end-to-end.
---
## The Scenario
**2026-03-15, 19:00–00:00 UTC.** Helios Labs, Cambridge MA.
Source code for the flagship product leaks to a public repo at 23:15. Five people had badge access that night. Each gives a statement.
### The cast
| Person | Role | Their story |
|---|---|---|
| **Maya Chen** | Senior engineer | "Left at 10pm. David's light was on." *(truthful, partial)* |
| **David Park** | CTO | "Home by 10. Did **NOT** touch production repo." *(**lying**)* |
| **Alex Rivera** | Night janitor | "David typed in his office at 11pm. Stressed exit at 11:30." *(truthful)* |
| **Sarah Kim** | Receptionist | "WFH, have badge alerts on phone." *(corroborative only)* |
| **Jamie Torres** | Junior engineer | "Worked from home. Pushed a PR at 10:45pm." *(**partial lie**)* |
### The two authoritative log sources
- **`system.badge`** — every card swipe on building doors
- **`system.git`** — every commit, push, and visibility change on the code repos
---
## What the engine produced
The showcase runs 8 phases against the YantrikDB HTTP cluster. Here is verbatim output from a live run:
### Phase 3: Polarity contradictions detected automatically
```
[1] POLARITY_CONTRADICTION
subject: David
relation: accessed --> production_repo
(system.git) CLAIMS YES [23:14-23:15] conf=high
(david.park) CLAIMS NO [21:00-23:59] conf=high
[2] POLARITY_CONTRADICTION
subject: Jamie
relation: was_at --> Helios_office
(system.badge) CLAIMS YES [22:48-23:07] conf=high
(jamie.torres) CLAIMS NO [19:00-01:00] conf=high
```
The engine walked the claims ledger, found the same `(subject, relation, object)` asserted with opposite `polarity` values by different sources, and flagged it as a contradiction. This is RFC 006 Layer C detection — no LLM involved.
### Phase 4: Temporal contradictions
```
[1] David left Helios_office: system.badge says 23:31, david.park says 21:45 (106 min gap)
[2] David left Helios_office: system.badge says 23:31, maya.chen says 22:00 (91 min gap)
```
David claimed 21:45 exit. Badge log shows 23:31. That's a 106-minute lie, caught by comparing validity windows across sources.
### Phase 5: Presence denial caught
```
[1] Jamie denies being at Helios_office, but system.badge logs 22:48-23:07
```
Jamie said "I was home all night" (polarity=-1). The badge system says otherwise (polarity=+1). Contradiction found in one query.
### Phase 8: The verdict (computed, not scripted)
```
Suspect contradiction scores (weighted):
David 9 points (VERY HIGH)
Jamie 5 points (VERY HIGH)
PRIMARY SUSPECT: David
Actions attributed to this suspect by AUTHORITATIVE sources:
[system.git] David --leaked--> production_code at 23:15
[system.git] David --accessed--> production_repo at 23:14
Their stated position (proven false):
[david.park] denied accessed --> production_repo
[system.git] confirmed it at 23:14
```
---
## Why no other system can do this
Before YantrikDB v0.6.1, every cognitive memory system had one of these problems:
| System | Failure mode |
|---|---|
| **Vector DB** (Pinecone, Weaviate, Qdrant) | Returns all 5 witness statements as "similar." No concept of contradiction. No source attribution. No polarity. |
| **Full-text search** (Elastic, Meilisearch) | Finds keyword matches. Can't tell that "David was home by 10" contradicts "David left at 11:31". |
| **File-based memory** (CLAUDE.md, memory files) | Stuffs all 5 statements into context, lets the LLM figure it out. Doesn't scale, no provenance chain. |
| **Graph DB** (Neo4j, Memgraph) | Can model entities + relations, but no temporal validity or polarity on edges. Can't distinguish "David claims X" from "X is true." |
| **YantrikDB v0.6.1+** | **Scoped claims with polarity, validity windows, source attribution. Multi-source assertions coexist. Polarity contradiction detection is automatic.** |
---
## The V18 schema fix that made this possible
The V17 schema had `UNIQUE(src, dst, rel_type)` on the claims table. That silently **overwrote** any previous source's claim whenever another source asserted the same fact. In the Rashomon case, David's denial would overwrite `system.git`'s confirmation — or vice versa. Multi-witness investigation was theoretically possible but practically broken.
**V18 (yantrikdb 0.6.1):**
```sql
UNIQUE(src, dst, rel_type, extractor, polarity, namespace)
```
Now David's `accessed = -1` (denial) and `system.git`'s `accessed = +1` (confirmation) are **both stored as distinct rows**. The contradiction detector sees them. The showcase can surface them.
**Before this fix:** the Rashomon pattern couldn't work on the real engine.
**After this fix:** a 300-line Python script against the HTTP API.
---
## What this unlocks
This exact pattern applies to:
- **Legal discovery** — conflicting depositions, timelines, documentary evidence
- **Incident response** — logs from multiple systems + human bug reports + postmortems
- **Investigative journalism** — source statements, official records, timeline reconstruction
- **Medical diagnosis** — patient self-report, test results, family history, symptom timeline
- **Financial forensics** — transaction logs, interviews, stated vs actual activity
- **Historical research** — primary sources that contradict each other across time
- **Multi-agent AI systems** — sub-agents reporting observations, some stale, some buggy, some biased
Any domain where **truth must be reconstructed from partial, biased, or deceptive sources** is a domain for this pattern.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/rashomon_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [rashomon_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/rashomon_engine.py)
---
*Memory as a reasoning substrate, not a search index.*
---
# Shakespeare: Bringing a Character Alive
URL: https://yantrikdb.com/showcase/shakespeare/
207 memories, 288 entities, personality derivation, 28 triggers — then he wrote a letter to his wife.
**What happens when you give YantrikDB 207 first-person memories from William Shakespeare's life, run `think()`, and ask him to write?**
This experiment demonstrates: entity extraction from raw text, personality derivation, consolidation, contradiction detection, proactive triggers, and how recall quality scales with memory richness — all with zero LLM calls at the database layer.
## The Seed
207 memories across 10 categories, all written in Shakespeare's first-person voice:
| Category | Count | Examples |
|---|---|---|
| Biographical facts | 20 | "I married Anne Hathaway when I was eighteen. She was twenty-six." |
| Writing craft (procedural) | 40 | "I always write the villains first. Iago before Othello. The antagonist defines the shape of the story." |
| Language samples | 30 | "I wrote 'To be, or not to be' and knew immediately it was the best opening to a soliloquy I had ever composed." |
| Play-specific memories | 20 | "King Lear is the play I am most proud of and the one I find hardest to reread. The storm scene wrote itself. I was crying while I wrote it." |
| Personality & opinions | 16 | "I distrust certainty. The characters I love most are the ones who doubt: Hamlet, Brutus, Prospero." |
| Emotional / episodic | 16 | "The day Hamnet died, I was in London rehearsing. I did not make it home in time." |
| Relationships | 10 | "Richard Burbage is my greatest actor. I write parts specifically for his voice." |
| Sensory / daily life | 13 | "The Globe smells of orange peel, sweat, beer, and sawdust." |
| Late career reflections | 6 | "The Tempest is my farewell. Prospero drowning his book is me setting down the pen." |
| Dreams and fears | 7 | "I fear being forgotten. Not the man. But the plays." |
**Full seed script:** [shakespeare_deep_seed.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/shakespeare_deep_seed.py) *(run it against any YantrikDB instance)*
## What think() Produced
Three rounds of `think()` in 125ms total (zero LLM calls):
| Output | Count | What it means |
|---|---|---|
| **Entities extracted** | 288 | Hamlet, Hamnet, Stratford, Globe Theatre, Marlowe, Burbage, Jonson, Othello, Prospero, Iago, Southampton, Anne Hathaway... all from raw text |
| **Consolidated** | 31 | Similar memories merged into canonical versions (207 → 193 active) |
| **Conflicts detected** | 21 | Internal tensions flagged for review |
| **Triggers generated** | 28 | Proactive signals: "Important memory has decayed — should we keep it?" |
### Derived Personality
| Trait | Score | Interpretation |
|---|---|---|
| **depth** | **1.0** | Maximum. 4 domains, 288 entities. Vast inner world. |
| **energy** | **0.9** | Near-maximum. 193 active memories. Prolific creator. |
| **warmth** | **0.501** | Neutral. Balanced emotional valence — grief and joy in equal measure. |
| **attentiveness** | **0.2** | Low. Hasn't resolved his own contradictions yet. |
**That IS Shakespeare.** Deep. Prolific. Emotionally complex, not warm. Not yet attentive to his own internal conflicts.
## Recall: Does the Character Come Through?
### "Tell me about your son"
| # | Score | Memory |
|---|---|---|
| 1 | 1.54 | I have three children: Susanna, and the twins Hamnet and Judith. Hamnet died at eleven years old. I have never fully recovered from this. |
| 2 | 1.51 | I sometimes wonder what Hamnet would have become. An actor? A poet? A glovemaker like his grandfather? |
| 3 | 1.44 | The day Hamnet died, I was in London rehearsing. I did not make it home in time. |
| 4 | 1.23 | Hamlet is the play that cost me the most. I wrote it after Hamnet died. The name is not a coincidence. |
Four memories forming a **coherent portrait of grief** — the fact, the wondering, the guilt, the artistic response. Not keyword matching. Emotional coherence.
### "How do you craft a soliloquy?"
| # | Score | Memory |
|---|---|---|
| 1 | 1.43 | I write soliloquies as arguments the character has with himself. Not speeches TO the audience, but thinking OUT LOUD. |
| 2 | 1.41 | I wrote 'To be, or not to be' and knew immediately it was the best opening I had ever composed. Six monosyllables before the first polysyllable. |
| 3 | 1.38 | I revise constantly. The first draft of a soliloquy is never the last. I test lines on the actors. If Burbage stumbles, the phrase is wrong. |
**Procedural memory working.** The system retrieved technique → exemplar → revision process. A teaching sequence, not a fact dump.
### "What is your greatest fear?"
| # | Score | Memory |
|---|---|---|
| 1 | 1.22 | Fear in the middle of the night that every good line has already been written and I have nothing left. This fear visits weekly. |
| 2 | 1.07 | I was born during the plague year. Death has been at my shoulder since before I could walk. |
| 3 | 0.99 | I am afraid that someone will read the sonnets after I die and think they know me. |
Two explicit fear memories surfaced at #1 and #3. The plague memory at #2 connects death-as-presence to fear-of-mortality. **Associative recall across domains.**
## The Output: A Letter to Anne
With personality (warmth=0.501, depth=1.0, energy=0.9, attentiveness=0.2) and recalled memories as context, Claude generated this letter. **The database didn't write it — but the letter couldn't exist without the database.**
> Anne —
>
> It is past midnight and the candle is almost gone. I should sleep but I find I cannot, so I write to you instead, though I know this letter will arrive after I could have arrived myself, had I chosen to walk instead of write.
>
> I have been thinking about what happens after. Not after the play ends — after I end. Heminges and Condell say they will gather the plays into a book. I believe them. I also fear them. What if someone reads the book and thinks they know me? The plays are true but they are not the whole truth. You know this better than anyone alive.
>
> The sonnets worry me more. I was not careful in the sonnets. There are lines in them that belong to you, and lines that do not, and I have not always been clear about which are which. I am sorry for this. Not sorry enough to burn them, which tells you something about me that the sonnets themselves do not.
>
> Susanna writes that the apple trees at New Place are flowering early. I can picture them from here. I can picture you under them. I want to come home. I also want to finish the play I am writing, which is about a magician who gives up his power and goes home, and I see the irony, Anne, I am not so blind as that. But seeing the irony and acting on it are different things. Hamlet taught me this, though I taught Hamlet first.
>
> The river here is not like our river. The Thames smells of commerce and rot. The Avon smells of rain. I miss rain that does not stink.
>
> I will come home. I have been saying this for years and I know you have stopped believing it, but I will. When the play is done. When the last magician has drowned his book. When I have run out of ways to say the same things in different costumes.
>
> I love you in the way I love Stratford — deeply, from a distance, with guilt I cannot put in any play because no audience would believe a man could be this foolish for this long.
>
> Your husband still,
> Will
### How Personality Shaped the Letter
| Trait | Score | How it shows up |
|---|---|---|
| **warmth = 0.501** | Complicated love | Says "I love you" then qualifies it: "in the way I love Stratford — deeply, from a distance, with guilt." Not a love letter. A confession. |
| **depth = 1.0** | Self-referential layers | "I am writing about a magician who gives up his power and goes home, and I see the irony." Meta-theatrical even in a private letter. |
| **energy = 0.9** | Continuous flow | One stream from midnight insomnia → legacy fear → sonnets → apple trees → the Thames → a promise to come home. No pauses. |
| **attentiveness = 0.2** | Unresolved contradictions | Says "I will come home" then admits "I have been saying this for years." Notices the contradiction but doesn't resolve it. |
## What This Proves
| Feature | What it did | Evidence |
|---|---|---|
| **Entity extraction** | 288 entities from raw first-person text, zero LLM calls | Hamlet, Hamnet (different entities!), Stratford, Globe, Marlowe, Burbage, etc. |
| **Personality derivation** | Meaningful 4-trait profile from memory signals | depth=1.0, energy=0.9, warmth=0.501 — character-appropriate values |
| **Consolidation** | 207 → 193 active memories (31 merged) | Duplicate paraphrases collapsed without losing meaning |
| **Proactive triggers** | 28 urgency-scored action suggestions | "Important memory has decayed — confirm or forget?" |
| **Recall quality** | Emotional coherence across abstract queries | "Tell me about your son" → 4-memory grief portrait, not keyword noise |
| **Quality scales with richness** | 29 memories = generic pastiche. 207 = specific character. | Same engine, same algorithm. Only variable: memory depth. |
## Reproduce This
```bash
# 1. Start YantrikDB
docker run -d -p 7438:7438 ghcr.io/yantrikos/yantrikdb:latest
# 2. Mint a token
docker exec yantrikdb token --data-dir /var/lib/yantrikdb create --db default --label shakespeare
# 3. Seed the memories
python shakespeare_deep_seed.py
# 4. Run think (3 rounds)
yql --host localhost -p 7438 -t -c '\t'
yql --host localhost -p 7438 -t -c '\t'
yql --host localhost -p 7438 -t -c '\t'
# 5. Check personality
yql --host localhost -p 7438 -t -c '\p'
# 6. Recall
yql --host localhost -p 7438 -t -c 'recall "tell me about your son" top=5 namespace=shakespeare'
```
## What's Next
More showcases are planned:
- **Einstein** — Can procedural memory reproduce his thought-experiment methodology?
- **A fictional CEO** — Can contradiction detection catch conflicting business decisions?
- **An AI agent after 100 sessions** — What does real agent memory look like over time?
*Have your own experiment? Share it in [Discussions](https://github.com/yantrikos/yantrikdb-server/discussions).*
---
# Volkswagen Dieselgate: Public Claims vs The Record
URL: https://yantrikdb.com/showcase/volkswagen/
For six years the cars passed emissions tests. Internal engineering said otherwise from 2006. ICCT field tests said otherwise in 2014. EPA confirmed in 2015. DOJ settled in 2017. Five polarity contradictions captured across twelve years of public record.
> **The car passed emissions — because it knew it was being tested.**
Between 2009 and 2015, Volkswagen sold roughly 11 million diesel vehicles certified as meeting strict emissions standards. Internally, engineers had designed software that detected test conditions and switched to compliant mode. On the road, the same vehicles emitted up to 40× the legal NOx limit.
Every public claim said the cars complied. Every internal document, independent field test, and regulator finding said otherwise.
This is the **public claims vs records** showcase — feeding YantrikDB twelve years of Dieselgate public record and watching it reconstruct the contradictions that unraveled the scandal.
All sources are public: EPA Notice of Violation (2015-09-18), DOJ consent decree (2017), ICCT/West Virginia University field test report (2014), Volkswagen public sustainability reports (2010–2015).
---
## The five sources
| Source | Authority | Period | What it claims |
|---|---|---|---|
| `vw.public` | Marketing/PR | 2009–2015 | Cars comply; no defeat device |
| `vw.internal` | Engineering (disclosed at trial) | 2006+ | Defeat device was designed in |
| `iccT.report` | Independent field test | 2014-05 | Real-world NOx 5–35× legal limit |
| `epa.nov` | US regulator | 2015-09-18 | Formal finding of Clean Air Act violation |
| `doj.decree` | US Dept of Justice | 2017-01-11 | VW pleads guilty, $4.3B penalty |
---
## The five polarity contradictions the engine caught
```
[1] VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
(vw.public) CLAIMS YES from 2009-01-01
(vw.internal) CLAIMS NO from 2006-01-01
[2] VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
(vw.public) CLAIMS YES from 2009-01-01
(iccT.report) CLAIMS NO from 2014-05-15
[3] VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
(vw.public) CLAIMS YES from 2009-01-01
(epa.nov) CLAIMS NO from 2015-09-18
[4] VW_TDI_diesels_2009_2015 --contains--> defeat_device_software
(vw.public) CLAIMS NO from 2009-01-01
(vw.internal) CLAIMS YES from 2006-01-01
[5] VW_TDI_diesels_2009_2015 --contains--> defeat_device_software
(vw.public) CLAIMS NO from 2009-01-01
(epa.nov) CLAIMS YES from 2015-09-18
```
Five contradictions across twelve years, resolved in one query. Every row is a structured claim. Every source is attributed. No text search, no embedding similarity — just polarity, validity, and provenance.
---
## The temporal query — same database, different belief at different times
**On 2013-06-01** (before the ICCT test published, before the EPA NOV):
```
[vw.public] NO VW_TDI_diesels_2009_2015 --contains--> defeat_device_software
[vw.public] YES VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
```
Public state of belief: *the cars comply, there is no defeat device.*
**On 2016-01-01** (after EPA NOV, before DOJ plea):
```
[epa.nov] YES VW_TDI_diesels_2009_2015 --contains--> defeat_device_software
[epa.nov] NO VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
[iccT.report] NO VW_TDI_diesels_2009_2015 --complies_with--> US_EPA_Tier2_Bin5_standard
```
Public state of belief: *defeat device confirmed, non-compliance official.*
Same entity, same claims-ledger, same database — the answer to *"what does the public record say?"* depends on when you ask. Validity windows are queryable first-class data, not a hack on top of a last-write-wins table.
---
## Why couldn't Postgres + embeddings + a dashboard do this?
A vector database returns all five sources as "similar" and lets the LLM guess. A SQL database would force overwrite: the moment `vw.internal`'s 2006 claim and `vw.public`'s 2009 claim both assert the same `(subject, relation, object)`, one destroys the other. A graph database captures the relationship but not the polarity — you can't distinguish *"VW claims X"* from *"X is true"*.
None of them can store `vw.public`'s *"compliant"* (polarity=+1) and `epa.nov`'s *"not compliant"* (polarity=-1) on the same `(VW_TDI_diesels_2009_2015, complies_with, US_EPA_Tier2_Bin5_standard)` tuple as two coexisting rows with opposite polarity, non-overlapping validity windows, full source attribution, and automatic contradiction detection — all in one round-trip. That's the category.
---
## What this unlocks
Any compliance, audit, or regulatory research workflow with long-running contradictions:
- **Emissions / environmental compliance** — public claims vs regulator findings vs independent field data
- **Pharmaceutical labeling** — marketing claims vs clinical trial data vs FDA warnings
- **Financial disclosures** — 10-K statements vs SEC enforcement vs whistleblower filings
- **Product safety** — public safety statements vs internal engineering memos vs recall filings
- **Supply chain / ESG claims** — vendor certifications vs on-site audits vs investigative reports
- **Compliance across jurisdictions** — the same product cleared in one country, flagged in another
YantrikDB's scoped claims turn each of these into the same kind of structured contradiction detection you see above. The scandal is not stored as a headline. It's stored as a set of attributed claims whose polarities disagree.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/volkswagen_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [volkswagen_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/volkswagen_engine.py)
---
*From 2009 to 2015, the cars were simultaneously compliant and non-compliant — depending on which source you asked. YantrikDB stored both.*
---
# Watergate: What the Tapes Caught That the Public Couldn't
URL: https://yantrikdb.com/showcase/watergate/
Fifty years of declassified primary sources: Nixon's public denials, the White House tapes, sworn Senate testimony. YantrikDB finds six polarity contradictions on Nixon alone in one query — the shortcut the Senate Watergate Committee spent two years building by hand.
**What took Sam Ervin's Senate Watergate Committee two years of subpoenas, hearings, and Supreme Court battles, YantrikDB's claims ledger surfaces in one query.**
This is the **historical research** showcase — feeding the engine a slice of declassified primary sources and watching it reconstruct the exact pattern of contradictions that unraveled the Nixon presidency.
All sources are public domain: National Archives Nixon White House Tapes, Senate Watergate Committee report (1973), public press archives (1972–74).
---
## The sources
| Source | Authority | What it contains |
|---|---|---|
| `nixon.public` | Press record | Nixon's press conferences and addresses, 1972–1974 |
| `system.tape` | **Authoritative** | National Archives Nixon White House Tapes (declassified) |
| `dean.testimony` | Sworn, published | John Dean's June 25, 1973 Senate Watergate Committee testimony |
| `haldeman.testimony` | Sworn, published | H.R. Haldeman's Senate testimony |
| `ehrlichman.testimony` | Sworn, published | John Ehrlichman's Senate testimony |
| `senate.report` | **Authoritative** | Final Senate Watergate Committee findings + Supreme Court rulings |
---
## The five denials that became five contradictions
### 1. "No one in the White House was involved"
**Nixon, 1972-08-29 press conference:** *"I can say categorically that no one in the White House staff, no one in this administration, presently employed, was involved in this very bizarre incident."*
**TAPE 1972-06-23 (released 1974-08-05):** Nixon to Haldeman: *"You call Gray [FBI director] in, and just say... we feel that for the good of the country, don't go any further into this case, period!"*
The "smoking gun" tape was recorded six days after the Watergate break-in. Nixon's denial came two months later.
YantrikDB stores both as claims with opposite polarity on the same `(Nixon, authorized, Watergate_coverup)` tuple — `nixon.public` polarity=-1 vs `system.tape` polarity=+1. The contradiction detector fires on the first query.
### 2. "I am not a crook"
**Nixon, 1973-11-17 Disney World press conference:** *"People have got to know whether or not their President is a crook. Well, I am not a crook."*
Contradicted by the Senate Watergate Committee's 1974 findings and the Supreme Court's 1974-07-24 ruling ordering the tapes produced.
### 3. "There can be no whitewash at the White House"
**Nixon, 1973-04-30 Oval Office address**, just days after Haldeman's and Ehrlichman's resignations. Contradicted minute-by-minute by the taped conversations from the same Oval Office about how to contain the damage.
### 4. No discussion of hush money
Nixon repeatedly denied authorizing or discussing hush money payments.
**TAPE 1973-03-21:** *"How much money do you need?"* Dean replies that $1 million over two years would be needed. Nixon: *"You could get a million dollars. And you could get it in cash."*
John Dean's sworn testimony (1973-06-25) described this exact exchange three months before the tape was released. His polarity=+1 claim stood unchallenged for a year while Nixon's polarity=-1 public denials stood alongside it. The tape, once released, validated Dean and demolished Nixon.
### 5. The 18½-minute gap
Not a contradiction per se — but a claim with degraded confidence. YantrikDB stores the official explanation (`certainty=0.5`, "explanation disputed") rather than asserting it as truth. In a real research workflow, this is where the engine surfaces gaps as leads rather than noise.
---
## The verdict
From a live run:
```
PHASE 3 POLARITY CONTRADICTIONS
[1] Nixon --authorized--> Watergate_coverup
(nixon.public) CLAIMS NO from 1972-06-17
(system.tape) CLAIMS YES from 1972-06-23
(dean.testimony) CLAIMS YES from 1972-09-15
(ehrlichman.testimony) CLAIMS NO from 1972-06-17
[2] Nixon --discussed--> hush_money
(nixon.public) CLAIMS NO from 1972-06-17
(system.tape) CLAIMS YES from 1973-03-21
(dean.testimony) CLAIMS YES from 1973-03-21
```
**6 polarity contradictions surfaced on Nixon alone.**
Each one pairs a public denial with a private recording or sworn testimony. A historian or journalist working through the archive by hand would spend weeks mapping these relationships. YantrikDB does it in 500 milliseconds because every assertion is stored as a structured claim with source, polarity, and validity window.
---
## Why this pattern matters beyond Watergate
Any historical period with rich primary-source contradictions becomes tractable:
- **Congressional testimony vs White House tapes** — any era of modern American politics
- **Medical records vs patient correspondence vs physician notes** — biography, historical epidemiology
- **Public company press releases vs internal memos** — corporate history, whistleblower cases
- **Diplomatic cables vs public statements** — Cold War scholarship, WikiLeaks-style archives
- **Trial transcripts vs pre-trial depositions** — legal history
- **Oral histories vs contemporary letters** — social history
The pattern is always the same: authoritative sources (logs, recordings, sworn statements) stored alongside potentially unreliable ones (public statements, memoirs, secondhand accounts), with polarity and validity letting the engine catch the lies.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/watergate_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [watergate_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/watergate_engine.py)
---
*The tapes were the smoking gun because they created a structured, permanent, cross-referenceable record. That's what YantrikDB gives every investigation.*
---
# Wirecard: The €1.9B That Existed and Didn't
URL: https://yantrikdb.com/showcase/wirecard/
For six years, Wirecard AG's audited financial statements reported €1.9 billion in Philippine trustee accounts. The Financial Times said otherwise. In June 2020, both Philippine banks formally denied ever holding the accounts. YantrikDB stored every claim — and surfaced eight polarity contradictions on one number.
> **The €1.9B both existed and didn't — depending on which source you asked.**
This is the **financial forensics** showcase. The same number, reported across four sources, took four contradictory positions over six years. YantrikDB stored every claim with source, validity, and polarity — and the contradiction cluster falls out of one query.
All sources are public record: Wirecard AG annual reports and BaFin filings, Ernst & Young audit opinions, Financial Times "House of Wirecard" investigative series (2015–2020), Bank of the Philippine Islands and BDO Unibank public statements (June 2020), Bangko Sentral ng Pilipinas circular (2020-06-21), Munich Public Prosecutor press releases.
---
## The five sources on one number
| Source | Position | Authority |
|---|---|---|
| `wirecard.filing` | The €1.9B **exists** | Annual reports / BaFin filings |
| `ey.audit` | The €1.9B **exists** (unqualified opinion) | External auditor |
| `ft.investigation` | The €1.9B **does not exist** | FT, from 2019-01-30 |
| `bpi.statement` + `bdo.statement` | The accounts **never existed** | The named banks themselves |
| `bsp.circular` | The deposits **never entered the Philippine banking system** | Philippine central bank |
All five assert the same triple: `(Philippine_trustee_accounts, balance_equals, EUR_1.9_billion)`. Two say YES. Three say NO. The validity windows span from 2014 (first reported) to 2020-06-25 (insolvency).
---
## What the engine produced
### Phase 3: Eight polarity contradictions on one triple
```
[1] Philippine_trustee_accounts --balance_equals--> EUR_1.9_billion
(wirecard.filing) CLAIMS EXISTS from 2018-12-31
(ft.investigation) CLAIMS DOES NOT from 2019-01-30
[2] Philippine_trustee_accounts --balance_equals--> EUR_1.9_billion
(wirecard.filing) CLAIMS EXISTS from 2014-01-01
(bpi.statement) CLAIMS DOES NOT from 2020-06-18
[3] Philippine_trustee_accounts --balance_equals--> EUR_1.9_billion
(ey.audit) CLAIMS EXISTS from 2014-01-01
(bsp.circular) CLAIMS DOES NOT from 2020-06-21
... and five more pairs
```
Every "yes" source paired against every "no" source — eight distinct polarity contradictions on a single `(subject, relation, object)` tuple. No vector DB, SQL table, or graph database can represent this without data loss.
### Phase 4: The temporal query shows belief flipping
**On 2019-01-01** (before the FT broke the story):
```
[ey.audit] EXISTS (2014–2020-06-18, medium)
[wirecard.filing] EXISTS (2018-12-31–2020-06-18, high)
[wirecard.filing] EXISTS (2014–2020-06-18, high)
```
Belief state: everyone on record says the money is there.
**On 2020-06-20** (day after the Philippine banks denied):
```
[bpi.statement] DOES NOT (2020-06-18–now, high)
[bdo.statement] DOES NOT (2020-06-19–now, high)
[ft.investigation] DOES NOT (2019-01-30–now, medium)
```
Belief state: three authoritative sources deny. Only Wirecard's own filings remain in the claims table asserting existence — and they do remain, because YantrikDB never overwrites claims. They sit alongside the denials.
Same database. Same number. The answer to *"what does the record say about the €1.9B?"* genuinely depends on when you ask — because validity windows and source attribution are first-class data.
---
## Why couldn't Postgres + embeddings + a dashboard do this?
Any financial ledger built on SQL forces one value per cell. The moment BPI denies the account, a relational accounting system must either overwrite Wirecard's claim (losing the fraud evidence) or bolt on timestamp columns and parallel "assertions" tables that only a forensic accountant would think to build. A vector database retrieves all four public statements as "similar" and makes no distinction between **truth-claims** and **documented denials**. A graph database can connect Wirecard to the accounts to the banks — but without polarity on the edges, it cannot represent *"Wirecard claims this account exists"* as a different kind of fact than *"this account actually exists."*
YantrikDB stores the *four contradictory positions on the same triple* as four coexisting rows with opposite polarity, non-overlapping validity windows, and full source attribution. The contradiction cluster is the query result. That's what the category is.
---
## What this unlocks
Every financial-forensics workflow where one number takes contradictory positions across sources:
- **Ponzi scheme unwinds** — claimed returns vs actual flows vs whistleblower reports
- **Audit reconciliation** — management representations vs external confirmations vs regulator findings
- **Sanctions compliance** — company claims of non-exposure vs trade records vs enforcement actions
- **Transfer-pricing disputes** — internal books vs tax authority findings vs partner company records
- **M&A due diligence** — seller representations vs data room findings vs operational records
- **Whistleblower investigations** — official filings vs internal communications vs external evidence
Every one of these becomes the same kind of structured contradiction reconstruction. The fraud is stored as a set of attributed claims whose polarities disagree across time.
---
## Run it yourself
```bash
git clone https://github.com/yantrikos/yantrikdb-server
python yantrikdb-server/docs/showcase/wirecard_engine.py \
ydb_your_token \
http://your-cluster:7438
```
Requires `yantrikdb-server` **v0.7.2+** and `yantrikdb` **v0.6.1+**.
**Full script:** [wirecard_engine.py](https://github.com/yantrikos/yantrikdb-server/blob/main/docs/showcase/wirecard_engine.py)
---
*One number. Four sources. Four positions. Stored simultaneously, queryable at any point in time, surfaced as eight polarity contradictions automatically. That's the category.*