I watched one among our engineers clarify the identical authentication sample to Claude Code for the fourth time final month. Not as a result of he forgot he’d defined it. As a result of the device forgot.
Each session, from scratch. “We use JWT validation on the gateway layer, not in particular person companies.” He’d mentioned it three days in the past. And the week earlier than. And each time he began a brand new session for the previous six months. Every time, the AI nodded alongside, adopted the directions completely, after which forgot all the things the second the session ended.
I saved excited about this, as a result of it felt just like the sort of downside that ought to already be solved. It’s 2026. These fashions are genuinely succesful. They’ll purpose about complicated codebases, debug refined race situations, write stable checks. And but they function with what I can solely describe as aggressive amnesia — a pathological incapacity to retain something previous the present session.
The autocomplete excuse
This made sense when AI coding instruments have been autocomplete engines. Copilot circa 2022 was finishing single traces of code. The context was one file. Why wouldn’t it want reminiscence? You kind, it suggests, you tab. Session reminiscence is irrelevant.
However that’s not what these instruments do anymore. We ask them to construct options throughout a number of information. Debug manufacturing points that require understanding system structure. Onboard new engineers to unfamiliar codebases. Run autonomously on GitHub points. And each single time, they begin from zero.
I hold coming again to this analogy: think about hiring an excellent contractor who exhibits up each morning with complete amnesia. They’ll code. They’re quick. However every single day you spend the primary hour explaining the challenge, the group conventions, the choices you’ve already made, the errors you’ve already realized from. And the following morning? Identical factor.
That’s the expertise proper now. For each group. With each device.
The issues that by no means make it into code
Right here’s what bugs me most. The stuff the AI retains forgetting isn’t within the code. It’s the stuff that lives between the traces:
Why we selected Postgres over DynamoDB. (Efficiency for our question patterns, but in addition as a result of the group has deep Postgres experience and nil DynamoDB expertise.)
Why the notification service is a monolith module and never a microservice. (We tried microservices. It was a catastrophe. We reverted in Q3 and no person documented why.)
That the billing pipeline has a recognized edge case the place occasions get silently dropped underneath excessive load. (Two engineers learn about this. One in every of them simply gave discover.)
None of that is in a file the AI can scan. A few of it was in a Slack thread from eight months in the past. Most of it’s in folks’s heads. And it’s precisely the sort of context that determines whether or not the AI’s output is right or subtly, dangerously improper.
“Simply put it in a config file”
I do know what you’re pondering, as a result of I assumed it too. CLAUDE.md. .cursorrules. System prompts. Simply write all of it down in a file and level the AI at it.
We tried. Everybody tries. And it really works — for about three weeks, till the file is stale and no person updates it as a result of updating a config file is upkeep work that doesn’t ship options. The one that wrote the unique file has moved on to different issues. New choices get made in conversations that by no means make it to the file. The file turns into a historic artifact that roughly corresponds to what the group believed in some unspecified time in the future up to now.
It’s the wiki downside another time. Somebody creates it with good intentions. It begins decaying instantly. Inside six months, builders actively mistrust it as a result of they’ve been burned by following outdated data.
“Simply use a much bigger context window”
The opposite well-liked reply. 200K tokens wasn’t sufficient, so now we have now 1M. Simply stuff all the things in.
I’ve spent a number of time excited about this, and I feel it essentially misunderstands the issue. An even bigger context window provides you extra room for the present session. It doesn’t provide you with reminiscence. It doesn’t inform you why the group made a selected architectural choice final quarter. It doesn’t know in regards to the manufacturing incident that formed how the group thinks about error dealing with. It doesn’t know that Sarah is the one one that understands the reconciliation pipeline.
An even bigger window is a much bigger scratch pad. The scratch pad nonetheless will get erased when the session ends. You haven’t solved amnesia — you’ve given the amnesiac a bigger pocket book that additionally will get burned each night time.
“Simply add extra brokers”
This one is newer and it’s the one which will get me. The reply to “the AI doesn’t know sufficient” is seemingly “add extra AIs that additionally don’t know sufficient, however give each a narrower job.”
A evaluation agent. A testing agent. A deployment agent. Fifteen specialised brokers, every with hardcoded directions for one process. Somebody on the group wrote these directions. Somebody has to keep up them. When the evaluation requirements change, somebody updates the evaluation agent. When the testing framework modifications, somebody updates the check agent. It’s the config file downside at the next degree of abstraction, with extra shifting components.
And what none of these brokers know? Something about your group. They know what somebody hardcoded into their directions. They don’t know what the group realized final week.
The query that retains nagging me
Right here’s what I hold coming again to: what if the AI simply… remembered?
Not the uncooked dialog transcript. That’s noise. However the precise data — the choices, the patterns, the errors, the conventions — extracted from conversations and out there in future classes. Not only for the engineer who had the dialog, however for the entire group.
An engineer explains why we use occasion sourcing for the audit system. That clarification turns into a structured data merchandise — out there to each different engineer, in each future session, with out anybody sustaining a file.
Somebody discovers a refined coupling between two companies whereas debugging. That discovery will get captured. Subsequent time somebody touches both service, the AI already is aware of in regards to the coupling. Not as a result of somebody remembered to doc it, however as a result of the system was listening when the data was created.
The AI that helped you debug a billing subject on Tuesday begins your Thursday session already figuring out what you found. The brand new engineer who joins subsequent month has an AI that is aware of all the things the group has realized up to now yr — from day one.
I take into consideration this loads as a result of it modifications what the device essentially is. It stops being a coding assistant and begins being organizational reminiscence. Not a wiki that somebody has to keep up. A residing data base that grows as a result of folks use the device.
The place this goes
The AI coding device market is about to separate. On one aspect: instruments that assist particular person engineers write code sooner. These are commoditizing. The fashions get cheaper each quarter. The wrappers get thinner. There’s no sturdy benefit.
On the opposite aspect: instruments that make a corporation’s collective intelligence out there to each engineer, each session, completely. These don’t exist but. Probably not. Not in a manner that really works.
I’ve been spending the final yr excited about what the second class appears like. The way you construct it. What the structure must be. The place the trade’s assumptions are improper.
All of it began from this one commentary: your AI device has amnesia, and no person appears to suppose that’s an issue price fixing.
I feel it’s the one downside price fixing.
