How Ramorie Works
Under the hood of the memory system — from remember() to find(), stage by stage.
A retrieval engine, not a note store
Most "AI memory" tools are a database with vector search bolted on. Ramorie is a multi-stage retrieval pipeline where every write goes through semantic indexing and every read goes through query expansion, hybrid ranking, entity graph lookup, propositional boost, and LLM rerank — with graceful fallbacks at every stage.
Your note is embedded into a 1536-dimensional vector, split into atomic claims for long-form memories, checked against similar existing memories for supersede candidates, and linked into the entity graph — all synchronously so the next find() sees it.
Your query is rewritten as a hypothetical answer (HyDE), embedded, then matched in parallel via pgvector cosine and PostgreSQL full-text search. Scores blend with entity bonuses and propositional hits, then an LLM reranks the top candidates.
Gemini slow? Timeout and return hybrid order. pgvector not installed? Lexical-only. Rerank parse fails? Keep the blended score. Retrieval never returns empty because of an infrastructure hiccup.
Memory pipeline — what happens when you remember()
Every remember() call runs four synchronous stages before returning. We chose synchronous over eventually-consistent so agents don't have to implement polling or retry loops — if remember() returns success, find() will see it on the very next call.
Persistence
The memory row is written with content, user/org scope, type auto-detected from content (decision, bug_fix, pattern, preference, reference, skill, general), tags, and visibility.
Embedding
Content is sent to Gemini's embedding-2-preview model with outputDimensionality=1536 (Matryoshka-truncated from the native 3072 to fit pgvector's ivfflat index cap). Timeout: 3s. Stored as a vector(1536) column.
Propositional split
For memories over 500 characters, an LLM breaks the content into atomic claims — one self-contained fact per row in memory_propositions. Each proposition gets its own embedding so find() can match a sharp claim buried in a long document.
Supersede detection
We scan for existing memories with cosine > 0.87. If any match, an LLM judges whether the new memory supersedes the old. On yes, the old memory gets superseded_by = new.id and is hidden from default find() results (audit trail preserved).
Entity extraction (projects, people, technologies named in the memory) happens async via a worker queue. Entities get linked into memory_relations so future queries can traverse the graph up to 3 hops.
Retrieval pipeline — what happens when you find()
find() runs up to ten stages. Most are cheap (SQL) and a few hit Gemini on the hot path — each with a hard timeout and a fallback so the agent never stalls waiting for an LLM.
| # | Stage | Latency |
|---|---|---|
| 1 | Intent classification Regex (EN + TR) buckets query into how_to / why / recent / owner / generic. Shapes type filter + purpose nudge. | <1ms |
| 2 | Auto-route check UUID / literal / single-token queries bypass HyDE entirely (ShouldAutoFastMode). Cheap path for lookup queries. | <1ms |
| 3 | HyDE expansion Gemini-2.5-flash rewrites query as a hypothetical engineer's note. 5-min LRU cache. Falls back to raw query. | 500-1500ms (0 on cache hit) |
| 4 | Query embedding Gemini embedding-2-preview → 1536-dim vector. Timeout 2s. Fail → lexical-only mode. | 300-800ms |
| 5 | Lexical scan PostgreSQL ts_rank on search_vector column. Supersede filter applied unless IncludeSuperseded=true. | 30-80ms |
| 6 | Semantic scan pgvector cosine distance over memories.embedding. Runs in parallel with lexical scan. | 50-150ms |
| 7 | Entity graph bonus Entities matching query terms get linked memories a +0.05 × 0.5^hop boost. Up to 3 hops via recursive CTE. | 20-80ms |
| 8 | Propositional boost Sharp-claim embeddings boost their parent memory by up to +0.10. Catches long-doc / short-claim mismatches. | 50-150ms |
| 9 | Blend & sort final = 0.55*semantic + 0.30*lexical + 0.10*recency + 0.05*usage + entity + propositional | <5ms |
| 10 | LLM rerank Top-20 candidates to Gemini for pair-wise scoring. Blended with original 0.7/0.3. In-process cache for repeat queries. | 800-2000ms (0 on cache hit) |
Reliability & fallbacks
Every Gemini call is wrapped with timeout + retry + circuit breaker. If the provider degrades, Ramorie degrades gracefully — never catastrophically.
| Stage | Fallback |
|---|---|
| HyDE | Use raw query for embedding |
| Query embedding | Switch to lexical-only (ranking_mode="lexical") |
| pgvector | Semantic scan returns empty, lexical continues |
| Rerank | Keep hybrid-blended order |
| Entity bonus | Skip bonus, base score used |
| Circuit breaker | Short-circuit calls, return ErrRateLimited |
Every Gemini-facing stage is controllable at runtime via env vars — FEATURE_HYDE_ENABLED, FEATURE_RERANK_ENABLED, FEATURE_SUPERSEDE_ENABLED, HYDE_CACHE_TTL_SEC, RERANK_TOP_K, RAMORIE_MODEL_<OP>, RAMORIE_TIMEOUT_<OP>_MS, RAMORIE_FALLBACK_MODEL_<OP>. You can disable any stage or swap its model without a redeploy.
Privacy & models
Today Ramorie uses Google's Gemini API for embeddings and LLM work. The abstraction layer is designed so that any op — embedding, HyDE, rerank, propositional split — can be swapped for a self-hosted model without touching call sites.
Your query text (on find) and your memory content (on remember). We never send your full memory corpus to the LLM — rerank only sees the top-20 titles + 400-char previews, never your full private data.
All vector math (pgvector cosine, lexical ts_rank, entity graph traversal, score blending) runs in your own Postgres. The LLM is only invoked for query expansion and reranking the already-retrieved candidates.
The aimodels registry (internal/aimodels/registry.go) lets you swap the embedding model to a locally-hosted open-weight model (BGE-M3, Qwen3-Embedding, Nomic) via RAMORIE_MODEL_EMBEDDING. When a provider abstraction ships, HyDE and Rerank will follow the same pattern. Embedding self-host is the highest-leverage swap — it accounts for 70-80% of Gemini calls.
Vault-encrypted memories (AES-256-GCM, client-side) never leak plaintext to the server. They're retrieved via the lexical path on server-side tokens + client-side decryption — LLM rerank is disabled for vault-encrypted projects by design.