AI Memory Startup Engram Raises $98 Million to Solve the Industry's Token Cost Bottleneck

Engram, a 13-person artificial intelligence startup founded just last October, has secured $98 million in a funding round backed by a roster of Silicon Valley heavyweights. The round includes participation from General Catalyst, Kleiner Perkins, Sequoia Capital, and OpenAI co-founder Andrej Karpathy. The massive capital injection for such a young company underscores the urgency of the problem Engram is trying to solve: the skyrocketing cost of running large language models in production environments.[1][7]

As businesses rush to deploy AI agents for complex, multi-step workflows, they are colliding with a fundamental economic bottleneck. Leigh Marie Braswell, a partner at Kleiner Perkins, described the current enterprise landscape as facing an explosion of data and an explosion of cost. Every time a user interacts with an AI, the system must process data, and those processing units—known as tokens—carry a direct financial cost that scales linearly with the length of the conversation.[1]

The root of this cost explosion lies in how modern AI models are architected. Standard large language models are inherently stateless, meaning they do not possess innate memory of past interactions. To maintain the illusion of a continuous conversation, developers must re-feed the entire chat history into the model's context window for every single new query. As conversations grow longer, the number of tokens processed per query balloons, driving up both latency and cloud compute bills.[3]

Engram’s architecture attempts to solve this by decoupling memory from the model’s active processing window. Instead of stuffing a massive, ever-growing chat history into the prompt, Engram acts as an intelligent, external memory layer. The system intercepts the conversation, extracts the core facts, and stores them in a structured database, allowing the AI to start fresh on each query while still having access to historical context.[2]

The technical mechanism behind this is what researchers call typed dense retrieval. According to a recent paper published on arXiv, the Engram system organizes conversational data into three canonical memory types: episodic, semantic, and procedural. When a user asks a question, a router searches this database and retrieves only the specific, top-ranked facts relevant to the immediate query, merging them into a compact evidence package for the language model.[3]

The efficiency gains demonstrated by this approach are staggering. The arXiv research indicates that the Engram architecture can match or exceed the performance of full-context baselines on long-horizon memory benchmarks while using approximately 1% of the tokens. By retrieving only the necessary context, the system drastically reduces the evidence budget required for the model to generate an accurate response. CNBC reports that the startup claims its models can match frontier labs while using up to 100 times fewer tokens.[1][3]

Beyond cost savings, this architectural shift addresses the notorious needle in a haystack problem that plagues traditional vector databases. Standard similarity searches often return contradictory or outdated facts—such as an AI retrieving both 'I live in Boston' and 'I moved to Vancouver' because both match the keyword search. Engram’s system detects these contradictions at write-time, supersedes the old fact, and maintains a clean version history, ensuring the AI agent isn't confused by its own memory.[2]

To appeal to cost-conscious enterprises, Engram has structured its business model around transparency. The platform operates on a bring-your-own-model basis, meaning it does not charge an inference markup. Customers pay a flat platform fee for memory storage and retrieval, while their actual AI inference goes through their existing, negotiated contracts with providers like OpenAI, Anthropic, or Google.[2]

Despite its small team and recent founding, Engram has already secured major enterprise customers. The company has signed deals with Microsoft, productivity software maker Notion, and the prominent legal AI startup Harvey. For these companies, integrating an efficient memory layer is critical; it allows them to offer highly personalized, long-term AI features to their users without absorbing catastrophic inference costs on the backend.[1][5][6]

The broader implications for the AI industry are profound. While frontier models are pushing toward massive million-token context windows, the sheer compute required to process that much data remains a physical and economic barrier. Industry analysts note that even as per-token costs slowly decline, the volume of tokens required for autonomous agents is growing much faster, making intelligent retrieval systems a mandatory component of the future AI stack.

This dynamic explains why venture capitalists are pouring money into this specific layer of the infrastructure. For AI software to transition from a novelty to a profitable, scalable business model, the unit economics must make sense. Investors view the token-cost bottleneck as an existential threat to AI margins, driving massive valuations for infrastructure startups that can mathematically prove they lower the cost of compute.[4][7]

By proving that structured, intelligent retrieval can substitute for brute-force context processing, Engram is charting a path toward sustainable AI. The ultimate goal is to enable AI agents that can remember a user's preferences, past workflows, and ongoing projects across years of interaction—all without generating a cloud bill that bankrupts the companies building them.[7]