Memory that discards nothing: storage, the key, and the order of layers

Origin. The principles of memory grew out of practice and a series of data-loss incidents; each incident left a rule. The material was assembled from the system's working rules and the dialogues in which those rules emerged.

Memory is not a warehouse. For a research system it is the foundation: unreliable data spoils even flawless work, and distortion in the foundation grows without bound, because each layer above inherits the error of the one below. So we hold several hard principles about memory, and nearly every one was hard-won through an incident.

First principle: do not discard

Any deletion in our system is replaced by a move to a separate "trash" zone that only the human clears. "Dead code," "duplicates," "obsolete" — all go there, nothing into oblivion. The reason is simple: the judgment "this isn't needed" is itself a source of error. Deletion is irreversible; moving costs nothing. Don't judge the need — just don't delete.

This has a partner principle at the loading level: load everything, filter at query time. During ingestion, all records are accepted — no thresholds, no culling. The urge to drop something is a reason to place a tag, not to delete. Once, filtering during ingestion caused us to lose 41% of our records: what looked like noise turned out to be needed for analysis. A filter during ingestion is irreversible; a filter at query time can always be replayed.

Second principle: the order of layers

Data lives in layers, and layers are not equal. Closer to the primary source — more reliable. A transcript of a conversation is more reliable than its paraphrase; the paraphrase more than a summary; the summary more than a note in memory. On conflict, the lower layer wins — the one nearer the source. The derivative is stale or distorted until checked against the source.

From this follows a working distinction between levels of consulting memory. Orientation (where did we discuss this? what happened?) — a paraphrase suffices, the error self-corrects at the next question. Foundation (a system change, a document, a rule that will be inherited) — a paraphrase is not enough; the primary source is required, and before you start building, not at the last step. The trap is that derivative memory looks authoritative — "it's our own data" — and the more convenient it is, the stronger the temptation to skip the source. Economics push toward the error; the rule holds it back.

Third principle: the key is forged at write time

Here is a working hypothesis we hold precisely as a hypothesis. If you store everything and never discard, the question of retrieval arises: how do you find the needed thing in an un-culled volume? Storage without retrieval is useless — un-retrieved data equals absent data ("if you forgot and didn't consult it, treat it as gone").

Our approach is closer to a Borges image than to a database with an index: store everything — and forge the key at the moment of writing. Not build one global index over a heap, but at each write lay in the links by which it will later be reached. Memory is then not a tree with one owner per file, but a graph: a record belongs to many relations at once. Moving files is thinking in a tree (one owner); setting cross-references is thinking in a graph. The neurostructure principle: don't move neurons, create connections.

This is a hypothesis about the right architecture, not a proven result. We name it that honestly.

Fourth principle: injectivity

A separate memory layer we build as injective — one where each element has a single, unambiguous inverse path back to its source. The point: from any derived record you can return to exactly what it was derived from, without loss and without guessing. Injectivity is a provenance guarantee at the level of structure, not of promise: not "we tried to preserve the source" but "the inverse mapping exists and is unique."

Why this is a feature of the system

Memory as foundation rather than warehouse; storage without discarding; the key at write time; the order of layers; injective provenance — together these yield the property everything is built for: the reliability of the work rests first on the reliability of the data. In research, where every error costs hours and sometimes tens of hours of investigation, this is not a luxury but a condition. Self-verification here is not a final checkmark but a primary discipline, calibrated by where the record lands: onto the foundation, always from the source; for orientation, a cheap check.

Boundaries

Honest caveats. "The key at write time" and the Borges image of memory are a working hypothesis about architecture, tested by practice, not declared solved. "Do not discard" and "load everything" are principles paid for by concrete incidents, but their cost (growth of stored volume) is real and demands its own discipline — tags and access layers, not deletion. Injectivity is a property we build a memory layer toward; where the inverse mapping is not yet guaranteed by structure, we mark that, rather than pass off the wished-for as the done.