Anky Memory System

Implementation report — February 15, 2026

What Changed

Anky now has a deep memory system. Before today, every writing session was analyzed in complete isolation — Anky had no idea who you were, what you'd written before, or what patterns run through your consciousness. That's over.

Now, after every 8-minute writing session, Anky:

1. Extracts structured memories from what you wrote (themes, emotions, patterns, breakthroughs, avoidances)
2. Embeds your writing as a vector for semantic search
3. Deduplicates against existing memories (so "fear of commitment" doesn't get stored 47 times)
4. Builds an evolving psychological profile
5. Recalls all of this the next time you write, injecting it into the reflection

The result: reflections that feel like they come from someone who actually knows you.

Backfill Results

All existing writing sessions were retroactively processed.

Profiles Generated

Architecture

How the memory system works under the hood.

  User writes for 8 minutes
           |
           v
  ┌─────────────────────────┐
  │   Writing Session        │
  │   saved to DB            │
  └──────────┬──────────────┘
             │
      ┌──────┴──────┐
      │             │
      v             v
  ┌────────┐   ┌────────────────────────┐
  │ Image  │   │ Reflection (SSE stream) │
  │ Pipeline│   │ + MEMORY CONTEXT       │  <── NEW
  └───┬────┘   └────────────────────────┘
      │
      │  (after image is done)
      v
  ┌──────────────────────────────────┐
  │   Memory Pipeline (background)   │  <── NEW
  │                                  │
  │  1. Embed writing (OpenAI)       │
  │  2. Extract memories (Claude)    │
  │  3. Store with dedup             │
  │  4. Update profile (every 5th)   │
  └──────────────────────────────────┘
    

The Memory Pipeline (runs after every anky)

Memory Recall (at reflection time)

When a user finishes writing and Anky streams back a reflection, the system now does this first:

Database Schema

Three new tables added to anky.db.

memory_embeddings

Vector storage for writing sessions. Each row = one embedded text.

user_memories

Structured memories extracted from writing. Deduplicated by semantic similarity.

user_profiles

Evolving psychological portrait per user. Updated every 5th session.

Files Changed

• src/memory/mod.rs NEW
• src/memory/embeddings.rs NEW — OpenAI embedding client, cosine similarity, vector storage/search
• src/memory/extraction.rs NEW — Claude Haiku memory extraction, semantic dedup, structured storage
• src/memory/recall.rs NEW — Builds memory context for injection into reflection prompts
• src/memory/profile.rs NEW — Psychological profile generation and evolution
• src/pipeline/memory_pipeline.rs NEW — Orchestrates the full memory pipeline + backfill
• src/db/migrations.rs MOD — 3 new tables + indexes
• src/config.rs MOD — Added OPENAI_API_KEY env var
• src/main.rs MOD — Added mod memory;
• src/services/claude.rs MOD — Memory-enriched reflection functions + streaming with context
• src/pipeline/image_gen.rs MOD — Spawns memory pipeline after anky completion + memory in fallback reflections
• src/pipeline/mod.rs MOD — Registered memory_pipeline module
• src/routes/api.rs MOD — Memory context in stream_reflection + backfill endpoint
• src/routes/mod.rs MOD — Registered /api/memory/backfill route

Cost

How It Feels to the User

Before: Every reflection started from zero. Anky was insightful but had amnesia. The 50th session felt exactly like the 1st — no recognition, no continuity, no "I've noticed you keep coming back to this."

After: The reflection system prompt now includes something like:

=== WHAT YOU KNOW ABOUT THIS PERSON ===
[Their full psychological profile — core themes,
emotional signature, growth edges, tensions]

=== RECURRING PATTERNS (significance: high) ===
- theme: building technology as a form of self-inquiry (seen 4x)
- pattern: oscillation between creation and surrender (seen 3x)
- breakthrough: recognizing that the product IS the practice

=== RELEVANT PAST WRITING MOMENTS ===
- "i keep coming back to this idea that the code is
  the meditation, not separate from it..."
- "today i realized that every time i try to control
  the outcome i lose the thread..."

=== HOW TO USE THIS CONTEXT ===
Reference patterns naturally. Notice evolution. Don't
just validate — show them what's changed since last time.
    

Claude reads all of this before writing the reflection. The result is a response that feels like it comes from someone who has been paying attention — because it has.

Inspiration: OpenClaw

This architecture was inspired by studying the OpenClaw project's memory system, which uses:

• Hybrid search (70% vector + 30% BM25 full-text)
• SQLite + sqlite-vec for vector operations
• Markdown memory files, chunked and embedded
• LLM-driven recall via tool calls

The Anky implementation adapts this for a fundamentally different use case: instead of chatbot memory (storing facts), it stores psychological patterns — the emotional undercurrents, recurring tensions, and growth edges that emerge across writing sessions. The deduplication is semantic rather than exact, and the profiles are interpretive rather than factual.

What's Next

The system is live and working. Potential future enhancements:

• Memory dashboard — Let users see what Anky remembers about them
• Cross-session growth tracking — "Here's how your writing has evolved over the last month"
• Memory in image prompts — Feed memory context into image generation for visual continuity
• Upgrade to sqlite-vec — Replace brute-force cosine search with proper ANN indexing (will matter at scale)
• Memory opt-out — Let users choose whether Anky remembers them