efficiency.love
← Field Notes
2026-08-10 · Build Notes · 6 min read

I got tired of my AI forgetting me. So I built it a memory.

Claude Code starts every session with amnesia. JayBrain is the memory system I built from scratch to fix that: a server with a real vector database, search that actually finds the right thing, and a sense of what is still true. More than 200 commits later, it runs the recall for my whole fleet of agents.

Here is the problem nobody mentions when they tell you AI is going to change your life. Every new session is a first day on the job. The assistant does not know who you are, what you decided last Tuesday, or that you already tried the exact thing it is about to confidently suggest. For a one-off question, fine. For a tool you work with every single day, that amnesia is a tax, and you pay it every morning before you have had coffee.

So in February 2026 I built the fix and I have not stopped since. It is called JayBrain, and the plain-English version is this: it is a memory that lives outside the AI and plugs into it through a standard socket. The socket is MCP (Model Context Protocol, the open standard that lets an assistant call outside tools). The memory is a database I built and tuned so that recall is fast, and more importantly, relevant.

what is actually under the hood
store → SQLite + sqlite-vec (a vector database)
embed → local ONNX model (no data leaves the box)
search → hybrid: meaning + keywords, blended
rank → freshness + importance, so today beats last month
serve → MCP server the assistant just calls it

That list looks tidy. Building it was not, and the interesting part is why. The clever-sounding bits were the easy bits. The hard bits were the ones that sound boring, which is almost always where the real engineering hides.

Finding the right memory is two problems, not one

Search by keywords and you miss the same idea worded differently: you wrote "the boss hates meetings," you search "manager," you get nothing. Search by meaning alone and you drag in everything vaguely related, a pile of "close enough" that buries the one line you needed. The answer is to do both at once and blend the scores. That is hybrid search, and getting the blend right is most of the game.

A memory that does not know what is stale will lie to you

This is the one people skip, and it is the one that matters most. If a memory treats a three-week-old note as exactly as true as something you wrote an hour ago, it will hand you yesterday's answer with total confidence. So every memory in JayBrain carries a sense of freshness and importance, and recent, load-bearing facts outrank old ones. A memory without that is not smart. It is just a very convincing way to be wrong.

Write to it every day and it becomes a hoarder

Give a system a memory and it will fill that memory with near-duplicates, the same fact in eleven slightly different outfits. So there is consolidation: clustering similar memories, deduplicating them, and archiving the ones that have gone cold. It is the difference between a library and a landfill.

And then the session crashes

Long-running work dies at the worst moment. So JayBrain keeps checkpoints, handoffs, and crash recovery: if a session drops, the next one picks up the thread instead of starting over. Unglamorous, invisible when it works, and the entire reason the thing is trustworthy.

The clever part is a database. The hard part was teaching it to forget the right things.

Why this is on the portfolio, not in a demo folder

Here is the point, if you are reading this to size me up. JayBrain is not a prompt. It is not a clever config or a wrapper around somebody else's API. It is a real piece of software: a server, a vector database, retrieval tuning, memory consolidation, and all the quiet plumbing that makes it run correctly every day instead of impressively once. People pay a monthly subscription for a worse version of this. I built mine, I use it every day, and it has been growing steadily for months, north of two hundred commits and counting.

That is the whole reason these notes exist. Anyone can claim they can build. This is me showing the actual thing, the design decisions, and the parts that were hard, receipts and scars included.

// Build Notes is the honest story behind the things in the build log. Real systems, real code, the boring parts left in on purpose.

One entry in a longer build log.

See the build log
// or reach me at hello@efficiency.love