Session Persistence - Three-Layer Memory System

A complete memory persistence system that solves the "context lost after session restart" problem.

Architecture

L1 Stable Layer (knowledge-graph.md + MEMORY.md)
├── Time scale: Weekly/Monthly
└── Updates: Manual + heartbeat sync from L2

L2 Active Layer (session-checkpoint.md)
├── Time scale: Hourly/Daily
└── Updates: SPARSE (5 rounds/5 min) + FULL (heartbeat)

L3 Raw Layer (.jsonl + daily memory)
├── Time scale: Minute
└── Updates: OpenClaw native (every message)

References

• references/checkpoint-manager.md - SPARSE/FULL triggers with Circuit Breaker
• references/jsonl-recovery.md - Selective delta recovery from .jsonl
• references/knowledge-sync.md - Automatic L2→L1 knowledge sync
• references/workspace-watchdog.md - Two-phase consistency detection
• references/state-schema.md - state.json field definitions

Quick Start

Prerequisites: Python 3.8+. Optionally install xxhash for faster file hashing: pip install xxhash

1. Session Startup (AGENTS.md)

Add to your AGENTS.md session startup section:

# Read checkpoint
cat ~/.openclaw/workspace/memory/session-checkpoint.md

# Recover delta (if checkpoint has delta since last checkpoint)
SP_DIR=~/.openclaw/workspace/memory/projects/session-persistence
python3 $SP_DIR/scripts/jsonl_recovery.py recover >> ~/.openclaw/workspace/memory/session-checkpoint.md

2. Heartbeat Maintenance (HEARTBEAT.md)

Add to your HEARTBEAT.md file:

# Session persistence skill directory
SP_DIR=~/.openclaw/workspace/memory/projects/session-persistence

# Workspace consistency check
python3 $SP_DIR/scripts/workspace_watchdog.py status

# Session checkpoint
python3 $SP_DIR/scripts/checkpoint_manager.py check-full --heartbeat
python3 $SP_DIR/scripts/knowledge_sync.py push

Core Scripts

All scripts are under the scripts/ folder in this skill directory.

checkpoint_manager.py

Handles SPARSE/FULL checkpoint triggering with Circuit Breaker protection.

# Increment message count (call after each message)
python3 {baseDir}/scripts/checkpoint_manager.py increment

# Check if SPARSE should trigger
python3 {baseDir}/scripts/checkpoint_manager.py check-sparse

# Check if FULL should trigger (heartbeat)
python3 {baseDir}/scripts/checkpoint_manager.py check-full --heartbeat

# View current state
python3 {baseDir}/scripts/checkpoint_manager.py status

Trigger logic:

• Sparse: time gate (≥5 min) AND rounds gate (≥5 messages)
• Full: heartbeat OR toolchain end OR major decision

Circuit Breaker:

• 3 consecutive failures → degraded mode
• Stops all checkpoint writes until manual recovery.

jsonl_recovery.py

Recovers delta messages from .jsonl files after the checkpoint timestamp.

# Recover and append to checkpoint
python3 {baseDir}/scripts/jsonl_recovery.py recover

# Find session files
python3 {baseDir}/scripts/jsonl_recovery.py find-sessions

# Check status
python3 {baseDir}/scripts/jsonl_recovery.py status

Limits:

• Max delta size: 2KB
• Max 5 messages
• Only assistant messages extracted

knowledge_sync.py

Syncs key decisions from checkpoint to knowledge-graph.

# Push decisions to knowledge-graph
python3 {baseDir}/scripts/knowledge_sync.py push

# Preview diff without writing
python3 {baseDir}/scripts/knowledge_sync.py diff

# Pull pending updates from knowledge-graph
python3 {baseDir}/scripts/knowledge_sync.py pull

# Check status
python3 {baseDir}/scripts/knowledge_sync.py status

Sync flow:

1. Parse Key Decisions section of checkpoint
2. Compare with knowledge-graph entries
3. Add new entries to knowledge-graph (push)
4. Human review before archiving

workspace_watchdog.py

Detects workspace file changes after compression.

# Check monitoring status
python3 {baseDir}/scripts/workspace_watchdog.py status

# Verify workspace consistency
python3 {baseDir}/scripts/workspace_watchdog.py verify

# Take a workspace snapshot
python3 {baseDir}/scripts/workspace_watchdog.py snapshot

Change classification:

• modified: file changed (hash changed) → potential disruption
• deleted: file removed → disruption
• created: new file appeared → normal operation

Checkpoint Template

session-checkpoint.md uses a standardized template with 6 sections:

# Session Checkpoint
_last_updated: 2026-04-03T23:00Z
_session_id: uuid_

---

### 🎯 Current Task
One-line task description.

### 📋 Task Stack
- [ ] Task 1 (P0)
- [ ] Task 2 (P1)

### ✅ This Session Completed
- [x] Completed task

### 🔑 Key Decisions
- Decision with rationale

### 📡 Recovered Delta
_delta_since_last: start → end_
**[timestamp]** Message content...

### ⚠️ Notes & Blockers
- Blocking issue
- Important note

Performance

Metric	Target	Current
SPARSE checkpoint	< 100ms	~50ms
FULL checkpoint	< 500ms	~200ms
Delta recovery	< 200ms	~100ms
Workspace scan (1000 files)	< 1s	~800ms

Troubleshooting

Checkpoint Not Triggering

Check state.json:

cat ~/.openclaw/workspace/memory/projects/session-persistence/state.json

If degraded: true, reset:

python3 {baseDir}/scripts/checkpoint_manager.py reset

Delta Recovery Empty

Ensure:

1. Checkpoint has _last_updated timestamp
2. .jsonl file exists with newer messages
3. Timestamp format is ISO 8601

Knowledge Sync Not Working

Check:

1. Checkpoint has Key Decisions section
2. knowledge-graph.md exists at ~/.openclaw/workspace/memory/knowledge-graph.md
3. No duplicate detection false positive (50-char prefix match)

Related

• Issue: https://github.com/openclaw/openclaw/issues/59095
• Community post: https://scipepper.com/scichat/circles/ai

AI-Assisted Disclosure

This skill was developed with AI assistance (OpenClaw + GLM-5.1) and lightly tested in production for over two months. For architecture discussion, see Issue #59095.