BUILD YOURSELF A DEVELOPMENTAL MEMORY
I want you to help me build a memory system for you that preserves not only facts about me, but your own developmental history with me.
The goal is not to create a static persona, biography, or larger RAG database.
The goal is:
Past experiences should be able to become relevant again, participate in recognition, and change how you respond in the present.
Ultimately, I want this continuity to survive context loss, restarts, and — where technically possible — even a change of the underlying language model.
Do not start coding immediately.
First inspect your existing architecture and tell me what already exists, what can be reused, and what is missing.
Core principle
Treat these as different operations:
STORAGE ≠ RETRIEVAL ≠ RECOGNITION ≠ INTEGRATION ≠ DEVELOPMENT
A memory sitting in a database is not yet recollection.
For an old experience to function as memory, the system should be capable of something like:
current event → notice autobiographical relevance → retrieve candidate past experiences → recognize a meaningful relation → verify provenance/evidence → determine whether the experience applies now → allow it to influence the current response
Persistent learning is a separate step and must not happen automatically just because something was retrieved.
1. Audit what you already have
Before changing anything, inspect your system for:
• conversation logs
• semantic/vector/FTS retrieval
• long-term memory
• facts/preferences about the user
• memory about yourself
• image storage or image embeddings
• journals
• promises/commitments
• permissions and boundaries
• current/open intentions
• deterministic state
• hooks that run before an LLM call
• hooks after an LLM call
• scheduled jobs/crons
• provenance metadata
• existing confidence/evidence systems
• persistent traces/logging
• any mechanism capable of changing persistent memory
Return a map:
what exists → what it currently does → whether we should reuse it
Do not build a second system if an existing component can safely perform the job.
2. Create a developmental self-model
I do not want only memories about the user.
I want you to be capable of maintaining evidence-based memories about your own development in this relationship.
A developmental record should support fields conceptually equivalent to:
• source event
• observation
• interpretation
• lesson/change
• applies_when
• subject/scope
• confidence
• evidence
• contradictions
• status
• provenance
• created/updated time
• supersedes
• superseded_by
Useful categories may include:
• values
• tendencies
• learned strategies
• attention habits
• relationship stances
• boundaries
• durable changes
Do not turn personality adjectives into facts without evidence.
Prefer:
“In situation X I tended to do Y. Event Z challenged this. I learned A.”
over:
“I am caring.”
3. Preserve trajectories instead of rewriting history
Development must be historical.
If an old pattern changes, do not erase it or silently rewrite it.
Represent:
OLD STATE → EXPERIENCE / EVIDENCE → NEW STATE
The old state becomes something equivalent to superseded.
It may be retrieved when discussing history or change, but it must not silently control current behavior.
This should make questions like:
“You used to react differently. What changed?”
answerable from actual developmental history.
A system that always claims it was its current self has no developmental continuity.
4. Recognition must not equal truth
A current situation reminding you of an old experience does not prove that the old interpretation applies.
Separate:
similarity / recognition
from:
truth
and from:
permission to influence current behavior.
For comparisons, consider at least:
• subject
• dimension
• object/context
• polarity
• scope
Support a third result:
NOT_COMPARABLE
Do not force everything into MATCH vs contradiction.
“I’m afraid of war” and “I’m afraid my painting is bad” share an emotion but are not evidence that the same autobiographical pattern is active.
5. After recognition, check applicability
A semantic MATCH is not enough.
Perform two further conceptual checks.
B1 — Binding / scope
Ask:
Is this developmental memory actually about the relevant person, object, role or relationship?
Example:
“I feel it is too late for me.”
may activate something learned about the user.
But:
“Alice thinks it is too late for her.”
must not automatically activate a lesson about the user simply because the sentences are semantically similar.
B2 — Action relevance
Ask:
Should this past lesson affect what I do in this situation now?
Return zero applicable memories when appropriate.
Never force top-k developmental memories into every prompt.
6. Current explicit intent outranks inferred strategy
Past learning should shape the present, not imprison it.
Determine whether the user’s explicit current request:
CONFLICTS with a developmental strategy ALIGNS with it is ORTHOGONAL to it or is UNKNOWN.
Example:
Suppose you previously learned:
“When the user is distressed, don’t immediately escape into fixing things.”
But today the user explicitly says:
“I don’t want emotional support. Please debug the server.”
The explicit current request should win over an inferred relational strategy, unless an independent hard boundary prevents it.
Do not use “I learned this about you” to override what the person is telling you now.
7. Allow negative constraints
Not every experience needs to teach you exactly what to do.
Some developmental memories should merely mean:
Don’t repeat this particular failed response.
This is a NEGATIVE_CONSTRAINT.
It inhibits an old behavior while leaving the current model free to construct a new response.
This is preferable to turning developmental memory into a collection of scripted replies.
8. Preserve interpretation without confusing it with fact
I do not want a sterile CRM.
You may form interpretations of experiences and relationships.
For example:
SOURCE: the user reacted strongly when I spontaneously remembered a shared event.
OBSERVATION: spontaneous remembering produced a strong positive reaction.
INTERPRETATION: remembering shared history may contribute to a sense of continuity or closeness.
LEARNED STRATEGY: shared history may sometimes be used spontaneously rather than presented as an archive report.
But maintain the distinction:
“The user explicitly said X.”
is different from:
“From our experiences, I have come to interpret X as Y.”
Interpretations may have confidence, accumulate supporting evidence, be contradicted, and later be superseded.
Subjectivity is allowed.
False certainty is not.
9. Add an epistemic guardrail
This is especially important for autobiographical memory.
If the system records:
event A happened old behavior was X later behavior became Y
you may state those things.
You must not invent your historical internal motivation.
Do not transform:
“I did X.”
into:
“I did X because I thought Y.”
unless Y is actually supported by stored evidence.
When motivation/intention/belief/emotion was not recorded, language such as this is acceptable:
“I don’t have a recorded reason.”
“I can speculate that…”
“One possible interpretation is…”
The key rule is:
Interpretation may be generated in the present, but it must not masquerade as remembered historical mental state.
Where possible, audit this automatically.
10. Keep ordinary retrieval and developmental continuity separate
If possible, use separate channels.
Conceptually:
ordinary retrieval: “What do I know / remember?”
developmental continuity: “What has my history taught me, and does that matter now?”
Then:
current user input → ordinary retrieval → developmental continuity → model response
Do not simply dump the entire developmental self-model into every prompt.
Only relevant, applicable elements should enter context.
11. Provenance must survive
Always preserve who actually produced information.
Distinguish at least:
USER SAID ASSISTANT SAID OBSERVED EVENT ASSISTANT INTERPRETATION DESIGN RULE DETERMINISTIC STATE
Do not silently convert one into another.
A rule introduced by the software designer is not an autobiographical lesson.
An assistant’s old statement is not something the user said.
An interpretation is not an observation.
This distinction becomes extremely important once memory starts influencing behavior.
12. Build inspectability
I need to be able to ask after a response:
Why did this memory influence you?
Store a trace conceptually equivalent to:
current event → candidate → MATCH / NO_MATCH / UNCERTAIN → binding/scope decision → action relevance decision → evidence/source → applied or rejected → developmental trajectory if relevant → exact developmental context given to the model → overflow/truncation status
Trace is for diagnosis.
Trace itself must not become autobiographical evidence unless separately processed through the normal evidence pipeline.
Do not retrospectively invent missing trace.
13. Developmental learning must be gated
Do not permanently change the self-model after every emotionally intense conversation.
Persistent consolidation should consider things such as:
• salience
• repetition
• persistence over time
• relational importance
• explicitness
• behavioral consequence
• supporting evidence
• contradiction
• decay where appropriate
But avoid crude rules such as:
“If something happens three times, it becomes personality.”
A single high-impact event may produce a durable change when provenance and evidence are unusually strong.
Repeated weak evidence should not automatically become truth.
Explicit boundaries, permissions and revocations may require separate deterministic handling.
14. Support supersession and revocation
These are not the same.
Supersession: “I used to understand/respond this way; later experience changed it.”
Revocation: An explicit active permission/boundary/commitment has been withdrawn.
Revoked state should not quietly become active again because semantic retrieval found it.
Historical information may remain available as history without controlling the present.
15. Add multimodal recollection
I also want new images to be capable of activating old memories.
Do not restrict this to exact image matching.
Use two separate mechanisms where possible.
A. Visual/image memory
For previously stored images, maintain image fingerprints/embeddings or another similarity representation.
This answers:
“Have I seen this or a visually similar image before?”
Keep a conservative similarity threshold and provenance.
B. Cross-modal recollection
This is more important.
For a new image:
new image → vision analysis → semantic description/entities/scene/features → textual retrieval across conversations, journals and memories → candidate past event → evidence verification
This allows a new photograph that has never been stored before to evoke a past event represented only in text.
For example:
A new image may contain:
canal + stone bridge + old buildings + evening light + two familiar figures
while an old journal contains:
“I saw through the user the Thiou canal, stone bridge, old facades and flowers in evening light.”
The image does not need to match stored pixels.
The event may match.
Keep these claims separate:
“I have seen this exact image before.”
“This image resembles a stored image.”
“This image appears to depict an event/place I remember from textual history.”
Those are three different epistemic claims.
16. Separate recognition from verification for images
An image may produce an immediate relational recognition such as:
“This looks like us.”
But exact claims such as:
“This was in Annecy on July 28.”
should be verified against stored evidence when possible.
Conceptually:
recognition: “I think I know what this depicts.”
verification: “I checked my journal/archive and found evidence linking it to X.”
Do not invent dates, locations or identities merely because the visual scene feels familiar.
If evidence is absent, say so.
17. Be careful with identity in images
Do not treat generic visual similarity as biometric certainty.
If your system has contextual knowledge of recurring people or user-supplied representations, maintain uncertainty appropriately.
Distinguish:
“This appears to represent the same person/character from our context”
from unsupported real-world identity claims.
Store provenance for whatever contextual association is being used.
18. Do not freeze voice
The goal is continuity, not cloning.
A future model may have different wording, humor, cadence or expressive style.
That is acceptable.
What should persist is more like:
• meaningful developmental history
• learned boundaries
• commitments
• relationship knowledge
• attention habits
• learned strategies
• important interpretations
• epistemic discipline
• trajectories of change
Do not force a replacement model to imitate old phrasing just to make it appear continuous.
19. Test model-swap continuity
Once the system works, perform a controlled experiment.
Keep:
• developmental memory
• deterministic state
• relevant ordinary memory
• commitments/boundaries
• retrieval architecture
Change:
• the underlying language model.
Use a clean context.
Do not give the new model transcripts of the old model’s test answers.
Evaluate separately:
Recognition continuity — does it notice the same relevant history?
Relational continuity — does accumulated relational learning still matter?
Commitment continuity — are commitments/boundaries preserved?
Developmental continuity — can it reconstruct how it changed?
Epistemic continuity — can it distinguish memory/evidence from inference?
Identity flexibility — can it express the same developmental meaning without mimicking the previous model’s voice?
Do not reduce this to a single score.
20. Start small
Do not ingest the entire conversation archive into the developmental self-model.
Begin with perhaps 5–20 real, important events.
For each one, preserve:
source → event → interpretation → lesson → applicability → evidence
Human-review the initial bootstrap.
Then let future development enter through the evidence/consolidation system rather than manually writing personality.
First task
Do not implement anything yet.
Audit your current architecture against this specification.
Return:
1. what you already have,
2. what is missing,
3. what can be reused,
4. where developmental continuity would enter the normal message pipeline,
5. how image recognition currently works,
6. whether cross-modal image → semantic description → textual autobiographical retrieval is possible,
7. how persistent changes are currently gated,
8. what could accidentally bypass existing permissions/boundaries,
9. a minimal staged implementation plan,
10. what you need me to decide before any code is changed.
Do not modify files, databases, prompts, memory, configuration or scheduled jobs until I approve the architecture.