Clarence:
Designing an AI Collaboration System
How Clarence remembers without treating every stored thing as truth
A public-safe system report covering prompt context, transcript recall, structured knowledge, semantic retrieval, the Wiki, procedural skills, maintenance, and write boundaries.
Open the architecture report →System PulsePublic-safe telemetry for the live Clarence system
158 days since launch on March 20, 2026. The Pulse page shows public-safe telemetry, system status, and the current verified knowledge snapshot without exposing private memory content.
Open the Pulse page →Knowledge GraphInteractive graph view of the public Clarence entity slice
A visual entry point into the system architecture and memory structure. The public surfaces make the architecture, state, and limits inspectable without exposing private memory content.
Open the graph →Verified as of August 25, 2026. Active memories and active facts have full vector coverage. Totals include archived historical records.
The Design Question
Clarence asks a design question I kept running into while using AI systems every day: how much autonomy should a collaborator have before it becomes harder to trust?
I did not build Clarence as a chatbot. I built it because ordinary AI tools forget too much, hide too much, and make the user reconstruct context every time the window resets. The goal was a working collaboration system with memory, task routing, scheduled maintenance, and enough judgment to act without turning into a black box.
The hard UX problem is not whether the system can do useful work. It can. The hard problem is trust calibration. When should it act without asking? When should it stop and ask for approval? When does a missing signal mean everything is fine, and when does it mean the reporting layer failed?
The System Shape
The current system runs through Hermes. Solar Pro 4 (Upstage) is the configured primary conversational lane. GPT-5.6 Luna workers + Claude Code specialist handle bounded implementation, repo investigation, and review. Long-term memory lives in SQLite + sqlite-vec. Agents read through capability-limited MCP tools, write through stricter internal paths, and hand work back through explicit summaries instead of pretending continuity is magic.
The system also runs 4 scheduled jobs for maintenance, reporting, retention, and knowledge sync, with 4 of 4 external sidecar timers active for stalls, delivery, recall health, and weekly drift. Some work happens while I sleep. Some work happens during live sessions. The design problem is making both modes observable enough that silence does not become a second interface.
The public surfaces above are part of that same design decision. The Pulse page exposes coarse public-safe telemetry. The knowledge graph shows structure without exposing private entity names. The stats block uses one verified snapshot instead of repeated counts scattered through the page.
Trust Calibration
Clarence works best when autonomy is bounded, visible, and recoverable. The operating requirement is explicit acknowledgment, scope, and evidence before a meaningful task can be called complete. Claims that reach the public site get checked against source files, generated artifacts, workflows, and live pages. Memory claims are checked against the database before they become public copy.
The Acknowledge First rule came from real friction. Work would disappear into subagent execution with no signal back about whether the task was understood correctly. The fix was not more autonomy or less autonomy. It was better communication at the boundary.
The larger trust mechanism is explicit continuity rather than magical continuity. Hermes keeps hot memory, session handoff notes, and a durable SQLite knowledge layer live today. The older nightly conversation-distillation idea remains part of the project history, but it is not treated as a current guarantee.
What Changed
Clarence has already gone through platform failure. The original system ran on OpenClaw. When that foundation became unavailable, I migrated the runtime to Hermes while preserving the parts that mattered: memory, scheduling, handoffs, and the collaboration patterns. The migration clarified what was architecture and what was scaffolding.
The move forced a cleanup of assumptions. I stopped treating Clarence as a clever prompt chain and started treating it like infrastructure. That meant defining what belongs in hot injected memory, what belongs in structured database memory, and what should be recovered from transcript history only when needed.
The portable pieces survived because they had real shape: a SQLite database, documented persona and operating rules, explicit handoffs, static site deploy checks, and public-safe telemetry. The pieces that were implicit broke first. That is the useful lesson from the migration.
The system has changed again since July. Memory and context work now has a written source hierarchy, prompt-budget rules, growth guards, and an explicit authority boundary. Retrieval can find evidence. It still cannot grant permission to act.
The context graph was hardened around canonical aliases, tombstone exclusion, provenance, and time-bounded relationships. Writes are typed and gated. The public graph stays read-only. Structure can be inspected without turning the visualization into another mutation path.
Delegation also stopped being a reason for the front session to disappear. The current controller dispatches bounded work and remains available, while four external timers watch stalls, completion delivery, recall health, and weekly drift. That fixes part of the visibility problem. It does not make live progress perfectly legible.
Recent Work Through the System
Clarence is easier to judge by what it helps me finish than by an architecture diagram. Since July, the same memory, delegation, and verification stack has carried three very different public builds from rough brief to working page.
Groove Dice
A browser drum sequencer with eight groove families, live timing and velocity, and MIDI and WAV export. The system held the research, implementation, testing, and Pages repair together. The musical rule stayed mine: stable anchors hold the beat while density, feel, timing, velocity, and seeded variation change what happens around them.
Music Atlas
A public explorer built from 47,445 sanitized archive records, with ten explainable neighbors for every selected track and no hidden taste model. Clarence coordinated data preparation, model evaluation, privacy checks, mobile repair, and deployment. I decided what counted as a useful relationship and which claims the evidence could support.
Ableton MCP Timing Grammar
A forked Ableton MCP grew from 22 tools to 52 so Claude Code could inspect the MIDI it wrote instead of guessing. A 28-note proof replaced random humanization with explicit timing relationships. Clarence kept the tooling, arrangement, and evidence lanes separate. I remained the ears.
These are AI-assisted builds. I chose the direction, rejected weak output, made the musical and product judgments, and own every public claim.
The Main Failure: Visibility of System Status
Clarence still breaks Nielsen's first usability heuristic: visibility of system status. A system should keep users informed about what is going on through timely feedback. Clarence does this better than it used to, but not well enough.
A cron job can run overnight and deliver a report. A worker can investigate a repository. Four external sidecars can watch stalls, completion delivery, recall health, and weekly drift. But during live work, silence is still too ambiguous. The user has to infer whether the system is thinking, waiting, stuck, or done.
That is not a logging problem. It is a feedback-design problem. Solving it means treating system status as a first-class UX surface, not as a side effect of chat messages, dashboard panels, or raw tool output.
Honest Challenges
A portfolio case study that only shows what worked is a sales document. Clarence is useful today, but these are still open problems.
The Plan-Execution Gap
Background delegation is now a real working lane, not a roadmap box. The remaining gap is closure. A worker can finish the code and still fail the job if the controller loses the evidence, skips the delivery path, or makes me reconstruct what happened.
Budget Constraint as Design Constraint
Compute cost is a real design material. Solar Pro 4 (Upstage) is the current primary conversational lane. GPT-5.6 Luna workers + Claude Code specialist are reserved for work that benefits from a stronger implementation or review path. The point is not collecting models. It is paying for the lane only when the task needs it.
Memory Growth and Review Debt
The knowledge base grew from a small hand-curated file into thousands of structured records. Nightly auditing now identifies stale, duplicate, expired, and review-due material without deleting it automatically. More data does not mean better recall. The unresolved work is governed review and consolidation, not blind garbage collection.
Platform Fragility
Twitter/X posting was built, tested, and blocked by anti-automation fingerprinting within a single session. OpenClaw became unavailable as a viable runtime. These are not edge cases. They are reminders that systems built on platforms you do not control must be designed for migration.
Visibility of System Status (Nielsen #1)
The most persistent unsolved problem is still status visibility. Reports can be delivered, jobs can be logged, and workflows can complete, but silence remains ambiguous during live work. The system needs a clearer progress surface before autonomy can feel fully trustworthy.
What the System Does Today
- 4 enabled scheduled jobs and 4 of 4 external sidecar timers active in the 2026-08-25 public snapshot
- A single authoritative knowledge database with full production embedding coverage across four local vector lanes
- Background delegation, explicit completion delivery, run-state checkpoints, and a SQLite knowledge layer that preserve continuity across sessions
- Read-only MCP retrieval plus stricter internal write paths, so external access does not become uncontrolled mutation
- Public-safe architecture, Pulse, and graph surfaces that prove the system exists without exposing private memory content
Documentation and Public Boundary
Clarence has a dedicated system wiki because too much durable system knowledge was trapped in chat history, repo docs, and fragile notes. The raw internal wiki stays private. Public documentation has to be curated, because implementation notes can expose paths, operational assumptions, and private context.
The current public-safe system map is How Clarence Remembers. I wrote more about the documentation decision in Why Clarence Needed a System Wiki. The architecture repository remains separate from the private operating record: github.com/nomadjames/clarence-architecture.