Feature Proposal: Optional Lazy Memory Analysis Mode for Token Efficiency

[orgoj]

Feature Proposal: Optional Lazy Memory Analysis Mode for Token Efficiency

Executive Summary

This document proposes an optional "lazy memory analysis" mode for claude-mem that would dramatically reduce token consumption by shifting from real-time AI processing to on-demand analysis. This would be an optional configuration, preserving the current default behavior for users who prefer it.

Current Behavior vs Proposed Alternative

Current (Default) Behavior

• Real-time processing: Every tool execution is immediately analyzed by Claude Agent SDK
• Automatic context injection: Sessions start with relevant memory context
• Proactive structuring: All observations converted to structured format immediately
• High token usage: Continuous AI processing throughout sessions

Proposed Optional: Lazy Analysis Mode

• Metadata-only storage: Store raw tool data + basic session metadata by default
• On-demand analysis: AI analysis triggered only when explicitly requested
• User-controlled context: No automatic context injection unless user asks
• Token-efficient: AI processing only for sessions that are actually searched/analyzed

Use Case Rationale

This feature targets users who primarily need:

1. Session search capabilities: Find which sessions covered specific topics
2. Compact session continuity: Context restoration when sessions are compacted
3. Optional detailed analysis: Deep dive only when sessions are genuinely relevant

Key insight: Many users don't need AI-structured memory for every session, only for sessions they actually revisit or search for.

Proposed Architecture (High-Level)

Configuration Option

{
  "env": {
    "CLAUDE_MEM_MODE": "default|lazy",
    "CLAUDE_MEM_AUTO_CONTEXT": "true|false",
    "CLAUDE_MEM_COMPACT_ANALYSIS": "true|false"
  }
}

Lazy Mode Behavior

1. Storage Changes (Lazy Mode)

• Store raw tool execution data (no AI processing)
• Capture basic session metadata: timestamps, file counts, user goals
• Maintain full-text search on user prompts only

2. Context Injection (Optional)

• Default: No automatic context at session start
• User-triggered: "claude-mem: give me recent context" command
• Compact sessions: Automatic analysis before context injection

3. Analysis Workflow

User Search → Find Relevant Sessions → Analysis Needed? → 
Yes: Analyze Session Data → Cache Results → Return Detailed Results
No: Return Basic Info

Feasibility Assessment

Technical Feasibility: High

• All required data is already captured by hooks
• Raw tool data storage is simpler than current AI processing
• On-demand analysis can reuse existing SDK infrastructure
• No fundamental architectural changes required

Benefits

• Token Cost Reduction: Potentially 80-90% reduction for typical usage patterns
• Better Performance: Faster session starts (no real-time AI processing)
• User Control: Users decide when they want memory analysis
• Storage Efficiency: Raw data is smaller than structured observations

Trade-offs

• Delayed Structuring: AI analysis happens after session completion
• Different User Experience: Less "magical" automatic memory
• Search Latency: First search of a session requires analysis time
• Implementation Complexity: Dual-mode behavior

Questions for Discussion

1. User Demand: Is there sufficient user demand for token-efficient mode?
2. Complexity: Is dual-mode behavior too complex for users to understand?
3. Migration: How to handle existing data when switching modes?
4. Default Settings: Should lazy mode be the new default for new users?
5. Feature Naming: Better naming for this concept than "lazy mode"?

Conclusion

This optional lazy memory analysis mode would provide significant token savings for users who primarily use claude-mem for session search and occasional deep dives, rather than continuous memory enhancement. The technical implementation appears feasible with minimal disruption to existing users.

The feature would be additive - all current functionality remains available, while offering a resource-efficient alternative for users with different usage patterns.

[thedotmack]

Lazy mode can be acheived by just text searching the transcript files though, active storage during session is not necessary, its all in the .jsonl files. This could be acheived with scripting... but it doesn't seem like there's a need? How would this provide any parity to Claude-Mem as it is now?

Don't mean to be overly harsh but just thinking about it from a memory generation perspective, that's the "third layer" in my original "layered memory system" idea I had.

Layer 1 = flat-file index of titles of records + where the data is stored
Layer 2 = that summarized data, the content associate with the title from index
Layer 3 = full transcript search

but I didn't need to implement layer 3, when layers 1 and 2 work so well

Then experimented with different DBs, did knowledge graph flat file memory and compared with performance from sqlite, it's like night and day. The biggest thing that ended up working better than i expected, was the content timeline. It evolved from "layered" then antrhopic dropped their Progressive disclosure skills docs.

But a key insight here is this, If you literally import the jsonl transcripts into a sqlite db, you can do full text search super fast. You can even chunk that literally by groups of a few words at a time, no semantic processing at all, import that in to a paralell chroma DB, and get some really great results. Not as great as the linked semantic data, but pretty damn impressive actually overall.

And a key insight from your message, while I don't agree that a full on lazy mode is right for claude-mem, I DO agree we need granular controls in the settings for context injection, etc. Users should have control over all the parts of it, with a clear description of what they gain or lose by enabling the feature.