When Claude Forgot: A Network Glitch, a CAP Theorem Detour, and the Data Structure That Avoids Conflict
This started as a Wi-Fi hiccup and ended with me redesigning a chatbot’s backend in my head.
The Glitch
I was mid-focus session, reporting progress to Claude, when my internet dropped for maybe ten seconds. Two or three of my messages vanished into the void. When the connection resumed, Claude picked up from an earlier point in the conversation and responded to my newest message as if the intermediate ones had never existed. It was like talking to someone who had sudden amnesia — confident, helpful, and working from incomplete information.
Most people would have been annoyed and moved on. I got curious.
What had just happened, architecturally? Claude didn’t crash. It didn’t say “I’m missing something.” It just kept going, cheerfully wrong. That behavior has a name in distributed systems: it chose availability over consistency. The system stayed responsive (available), but at the cost of having a complete, coherent view of our conversation (consistent).
That’s one side of the CAP theorem — the result that says a distributed system experiencing a network partition must choose between consistency and availability. You can’t have both. And Claude, or rather the infrastructure behind it, apparently chose AP.
Was That the Right Call?
My immediate reaction: no. The cost asymmetry is wrong. A few seconds of “reconnecting…” is a minor annoyance. But a misunderstanding mid-conversation — especially one where I’m building on prior context — forces me to backtrack, re-explain, and lose the thread of what I was working on. The recovery cost from inconsistency is much higher than the cost of brief downtime.
Most mature messaging systems agree with me. WhatsApp, iMessage, Slack — they all queue messages during partitions and deliver them in order. They show you “sending…” rather than silently dropping messages and pretending everything’s fine.
But then I had to be honest with myself: I can’t actually prove it was an intentional AP choice. Maybe it was just an emergent behavior of how the pieces fit together. And that question — how do the pieces fit together? — pulled me down the rabbit hole.
What’s Behind the Curtain
Here’s the thing most people don’t realize about LLM chatbots: the model has zero memory. Claude doesn’t remember anything between calls. Every time I send a message, the system reads my entire conversation history from a database, stitches it into a prompt, and ships it to the model. “Memory” in an AI chatbot is a database problem, not an AI problem.
The architecture is roughly: client sends a message → API gateway receives it → conversation store persists it → prompt assembler reads the full history and builds the context → LLM processes whatever it receives and streams a response back.
My glitch almost certainly happened at the first link in that chain. My browser rendered the messages optimistically (showed them as “sent” before the server confirmed), but the server never received them. When the connection resumed, the server’s conversation store — the source of truth for prompt assembly — was missing my intermediate messages. Claude responded based on what the server actually had, which was an incomplete history.
The LLM itself is blameless. It processes whatever context it receives. It has no way to know something is missing. A human friend would say “wait, I think I missed something.” Claude, by design, will just barrel forward confidently with incomplete information. The AP choice combined with Claude’s tendency to be helpful-no-matter-what creates a compounding failure mode.
The Local-First Temptation
My first instinct was: why not store the conversation locally? It’s just me and Claude. It doesn’t need to be in some distributed database.
But that instinct is wrong, for a reason that took me a minute to see. The LLM runs on Anthropic’s servers. The prompt assembly happens server-side. If the store were local-first, my client would need to upload the entire conversation history with every single message — potentially tens of thousands of tokens, redundantly, every time. That’s how the raw API works for developers, but it doesn’t scale for a consumer product with long conversations.
And then I realized I have Claude on three devices — my phone, my MacBook Pro, and my MacBook Mini. Even if local-first were efficient, I’d need all three devices to stay in sync. That’s four nodes (three clients plus the server) that all need a coherent view of the same conversation. I’ve already got a distributed system whether I wanted one or not.
There’s also a constraint I can’t design around: Anthropic needs server-side access to conversation history for safety and moderation. If the canonical store lived on my device, the server would be trusting the client to send an accurate, unmodified history. That’s a security problem.
So the architecture is almost certainly: server-side canonical store, client-side cache for display. My glitch happened in the gap between what my client showed me and what actually made it to the server.
The Conflict Problem
But the multi-device scenario opened a new question. What if I’m on my MacBook, my internet drops, and I send a message from my phone at the same time? Now there are two divergent conversation histories. When they sync, which one wins?
This is where I finally got to explore something I’d heard about but never fully understood: CRDTs — Conflict-free Replicated Data Types.
The name tells you the trick. These aren’t data structures that resolve conflicts. They’re data structures designed so that conflicts can’t happen in the first place. The merge function is baked into the data structure itself, and it satisfies three mathematical properties — commutativity, idempotency, and associativity — that guarantee every replica converges to the same state, regardless of the order updates arrive. It’s a mathematical proof, not a best-effort heuristic.
A Family, Not an Algorithm
CRDTs aren’t one thing. They’re a family of data structures, each avoiding conflict in a different way.
A G-Counter (grow-only counter) is the simplest. Each device keeps its own counter. The merge is: take the max of each entry. Since counters only grow, there’s nothing to conflict. Good for page views, like counters, event tallies.
A PN-Counter handles the case I was most curious about: what if one device increments and another decrements? The answer is beautifully indirect — you don’t resolve opposite operations by picking a winner. You keep two G-Counters internally, one for additions and one for subtractions. The real value is P minus N. Both sub-counters grow monotonically, so both are conflict-free. The opposition is decomposed away.
An OR-Set (Observed-Remove Set) allows both adding and removing elements. Each element gets a unique tag on add. Remove only deletes tags the device has seen. If another device concurrently re-adds the same item with a new tag, the re-add wins. It’s a design choice — add wins over concurrent remove — but it’s deterministic and convergent.
A LWW-Register (Last Writer Wins) is the most blunt. Each device stores a value paired with a timestamp. Higher timestamp wins. Simple, but lossy — the overwritten value is gone forever.
The Right CRDT for a Chat
So which one fits a conversation? I think it’s the G-Set — the grow-only set. And the reason isn’t just simplicity. It’s semantic correctness.
Chat messages have causal dependencies. Message 5 only makes sense because of messages 1 through 4. Deleting message 3 doesn’t just remove data — it breaks the semantic chain. The data structure should reflect that reality: messages are immutable, append-only facts. A conversation is a grow-only log.
Even when chat apps offer a “delete message” feature, it’s typically a soft delete — a tombstone flag that hides the message from display without removing it from history. The underlying data structure can still be a G-Set with a display filter on top. The append-only log remains intact.
The Loop
So here’s the full chain: a ten-second internet glitch → a question about AP vs CP → the realization that “memory” in an AI chatbot is a database problem → the discovery that multi-device sync makes it a distributed systems problem whether you want it or not → CRDTs as the right abstraction for the conversation store → G-Set as the specific variant, because conversations are semantically append-only.
What I find satisfying is that the answer to my original question — “was AP the right choice?” — turned out to be: probably nobody chose it. It was an emergent behavior of the gap between optimistic client rendering and server-side truth. The system wasn’t designed to choose AP in that moment. It just fell into it because the pieces weren’t coordinated for that edge case.
And the fix isn’t to “choose CP” as a policy. It’s to design the data layer so that the question doesn’t arise — queue messages during partitions, deliver them in order, merge deterministically. Use a CRDT. Avoid the conflict entirely. Don’t resolve it; dissolve it.
That’s the CRDT philosophy in one sentence. And it’s a pretty good philosophy for more than just distributed systems.
Full Conversation — the unedited exchange that produced this post
Me: Today I had an interesting moment with Claude. I was reporting my focus session and my progress, and there was a glitch on the internet. Two or three of my messages were lost. When the internet resumed, Claude continued from an earlier endpoint and took in my newest message, which caused some misunderstanding. It’s like Claude has suddenly lost memory in the past two minutes. This is interesting. This means Claude chose AP over CP in the CAP theorem. Is AP over CP really the right design in this scenario?
Claude: Your CAP framing is roughly right. During the network partition, the system prioritized keeping Claude responsive (A) over ensuring Claude had the complete, ordered message history (C). The result: Claude confidently responded to your latest message but with a stale view of the conversation.
CP might actually be the better design here. The cost asymmetry matters — a few seconds of unavailability is a minor annoyance, but a misunderstanding mid-conversation forces you to backtrack and re-explain. Most mature messaging systems lean CP — WhatsApp, iMessage, Slack all queue messages during partitions and deliver them in order. Conversational AI has a unique vulnerability: a human friend would say “wait, I think I missed something,” but Claude will barrel forward confidently with incomplete information.
Me: I would absolutely choose CP over AP. It’s annoying and time-costing to re-explain things. That costs time, which may be longer than the time of the internet glitch. Definitely CP over AP. Conversations need to be consistent.
Claude: That’s a solid cost analysis. Consistency failures in conversation have asymmetric recovery costs — the glitch itself might be 10 seconds, but the cleanup from a misunderstanding can be minutes of re-explaining, plus the cognitive disruption. A conversation isn’t a series of independent requests — it’s an accumulating shared context. Dropping messages isn’t graceful degradation, it’s corruption.
Me: I wonder if there are other considerations for this scenario. Maybe the system architecture, the way the data is stored, the way they process requests. I would like to take this as an opportunity to learn how the backend of an AI chatbot with LLM works. I can’t prove that they intentionally chose AP over CP — maybe that was just my misunderstanding. But I’d like to figure out the options at the backend so I can learn system design architecture from it.
Claude: The foundational insight is that the LLM itself is completely stateless — it has zero memory between calls. Every time you send a message, the system reads your entire conversation history from a database, stitches it into a prompt, and sends that to the model. “Memory” in an AI chatbot is really a database problem.
The architecture is roughly: client → API gateway → conversation store → prompt assembler → LLM inference → response stream back. There are two main failure points. First, between client and server — messages lost in transit during a network partition. Second, between the store and prompt assembly — if the system uses eventual consistency, a message could be written to one replica but the prompt assembler reads from another that hasn’t synced yet.
Your glitch almost certainly happened at the first failure point. Your browser sent messages that never got acknowledged. The client showed them optimistically, but the server never received them.
Me: I guess the conversation store should be at the local machine, right? Because it’s only about me and Claude. But I have Claude on my cell phone, my MacBook Pro, and my MacBook Mini, so messages need to be synchronized. Still, you can prioritize saving to your local device and then back up to a cloud.
Claude: There’s a key architectural constraint: the LLM runs on Anthropic’s servers, so the conversation history needs to be accessible server-side for prompt assembly. If the store were local-first, every message would require uploading the entire conversation history — that’s how the raw API works, but it doesn’t scale for a consumer product.
Your multi-device observation is exactly why this has to be a distributed systems problem: you have three devices plus the server — four nodes that all need a coherent view of the same conversation. And Anthropic needs server-side access for safety and moderation, so the canonical store must be server-side.
Your instinct about local-first with cloud backup is called local-first software — apps like Notion and Apple Notes use variations. The tricky part is conflict resolution when you send messages from different devices simultaneously.
Me: I should explore the idea of CRDT. I heard the word but wasn’t clear about it. What is it? Is it a strategy to resolve conflicts?
Claude: CRDT stands for Conflict-free Replicated Data Type. It’s a data structure designed so that multiple copies can be modified independently, and when they sync up, they automatically merge into the same result — no conflicts, no manual resolution, no central coordinator. The secret is that the merge function has three properties — commutativity, idempotency, and associativity — which guarantee convergence regardless of order.
For a chat conversation, you could design a CRDT where each message is an element with a timestamp and unique ID. The merge rule is set union, sorted by timestamp, deduplicated by ID. Both devices independently compute the same result.
The tradeoff: CRDTs give you eventual consistency with guaranteed convergence, but “eventual” is the key word. During the partition, each device has a different view. CRDTs solve the merge problem but not the “should I serve a response before all replicas have converged” problem.
Me: I thought CRDT resolves conflicts detected when two events happened concurrently. When you describe how CRDT works, it applies changes in order of the timestamp. How can you use a timestamp to decide order if they have the same time?
Claude: You’ve found a real flaw in my explanation. There are two separate problems: merging the data (making sure both devices end up with the same set) and display ordering (deciding what order to show them in).
For chat, the CRDT is essentially a grow-only set (G-Set). The merge operation is just set union — no ordering involved. That’s why it’s “conflict-free.” You’re not resolving a conflict, you’re avoiding one entirely.
Timestamps alone are unreliable for three reasons: clock skew between devices, identical timestamps at fine granularity, and causality not being captured. Vector clocks give you a partial order — they can tell you “D happened before E” or “D and E are concurrent” — but when concurrent, you need a deterministic tiebreaker like device ID. The tiebreaker is arbitrary but consistently applied everywhere.
Me: So CRDT is not an algorithm — it’s a strategy with different rules. As long as those rules have commutativity, idempotency, and associativity. You mentioned grow-only set. Are there other types? What if one device made an operation and another made the opposite — like incrementing vs decrementing?
Claude: That’s your exact question answered by the PN-Counter. One device does +1, another does −1. There’s no winner — both operations are preserved. The trick is decomposition: instead of one counter that goes up and down, you keep two grow-only counters — one for increments, one for decrements. The final value is P minus N. Since each sub-counter only grows, each is conflict-free.
The deeper design principle: you don’t resolve opposite operations by picking a winner — you redesign the data structure so that opposite operations don’t conflict in the first place.
Other CRDT types: G-Counter (grow only), PN-Counter (decomposes opposites into two growing quantities), OR-Set (uses unique tags so add-wins over concurrent remove), LWW-Register (picks latest timestamp, lossy but simple).
Me: I think for messages, you have to go with G-Set. A message can’t be deleted because what happened in previous messages already had an effect on the following messages.
Claude: That’s strong reasoning. Chat messages have causal dependencies — message 5 only makes sense because of messages 1 through 4. Deleting message 3 doesn’t just remove data, it breaks the semantic chain. The data structure should reflect that reality: messages are immutable, append-only facts. G-Set is the natural fit.
You’re not picking G-Set because it’s the simplest — you’re picking it because conversations genuinely are grow-only at the semantic level. The restriction isn’t a limitation, it’s an accurate model.
Even when chat apps offer “delete message,” it’s typically a soft delete — a tombstone flag. The underlying data structure can still be a G-Set with a display filter.