Letta, one of the most anticipated AI startups at UC Berkeley, has just come out of hiding

A startup called To read has just come out of hiding with technology that helps AI models remember users and conversations. Built in the famed startup factory of UC Berkeley’s labs, it also announced $10 million in seed funding led by Felicis’ Astasia Myers, at a post-money valuation of $70 million.

Letta also advantages from the backing of some of the most outstanding angel investors in AI, including Jeff Dean of Google, Clem Delangue of Hugging Face, Cristóbal Valenzuela of Runway and Robert Nishihara of Anyscale.

Founded by Berkeley PhD students Sarah Wooders and Charles Packer, it’s a highly anticipated AI startup. That’s since it’s the brainchild of Berkeley’s Sky Computing Lab and is a business arm of the popular open-source project MemGPT.

Berkeley’s Sky Computing Lab, led by renowned professor and Databricks co-founder Ion Stoica, is the offspring of RISELab and AMPLab, which spawned corporations corresponding to Anyscale, Databricks, and SiFive. In particular, Sky Lab spawned quite a few popular open-source large language model (LLM) projects, including Gorilla LLM, vLLM, and the LLM structured language SGLang.

“A lot of projects came out of the lab very quickly, within a year. Just people sitting next to us,” Wooders described. “So it was an amazing time.”

One such project is MemGPT. It is so popular that it went viral even before its release.

“Someone beat us to it,” Packer told TechCrunch. The founders had published an information document on Thursday, Oct. 12, 2023, and planned to publish a more detailed document and code to GitHub the following Monday. Some random person found the document, posted it to Hacker News on Sunday, and “it went viral on Hacker News before we had a chance to properly publish the code or publish the document or, like, start a Twitter thread or something like that,” he said.

The reason for the excitement was that MemGPT alleviates a pernicious problem for LLM: In their native form, models like ChatGPT are stateless, meaning they don’t store historical data in long-term memory. That’s problematic for AI applications that depend on learning from and understanding a user over time—from customer support bots to apps that track health care symptoms. MemGPT manages data and memory so AI agents and chatbots can remember previous users and conversations.

The newspaper post stayed at the top of Hacker News, a preferred developer site run by Y Combinator, for 48 hours, Packer said. So he spent the weekend and the next few days answering questions on the site, attempting to get the code ready for release. Once the project was continue to exist GitHub, a link to it went viral on Hacker News, again. Interviews on YouTube and tutorials, Medium posts, 11,000 stars and 1.2k forks on GitHub happened quickly.

Myers of VC Felicis also learned about Wooders and Packer while reading about MemGPT and immediately realized the business possibilities of the technology.

“I saw that paper when it came out,” she told TechCrunch, and immediately reached out to the founders. “We had an investment theme around AI agent infrastructure, and we appreciated that a really important piece of that was managing data and memory to make these conversational chat bots and AI agents effective.”

The founders continued to virtually drive down Sand Hill Road, on Zoom calls with enterprise capitalists before selecting the one who first loved them.

In the meantime, Stoica was brokering connections with Dean, Nishihara, and other outstanding Silicon Valley angel investors. “A lot of the Berkeley professors, just by virtue of being at Berkeley, are very well-connected,” Packer recalled, describing how easy the angel investor process was. “They have their eye on projects coming out of this lab that are going to be commercialized.”

Competition and the threat from OpenAI o1

While MemGPT is already available and in use, the business version of Letta, Letta Cloud, isn’t yet open for business. As of Monday, Letta is accepting requests from beta users. It will offer a hosted agent service that enables developers to deploy and run stateful agents in the cloud, accessible via REST APIs, a programming interface that may maintain state. Letta Cloud will store the long-term data vital for this purpose. Letta can even offer developer tools for constructing AI agents.

With MemGPT, Wooders sees a wide selection of applications. “I think the most common use case we see is basically highly personalized, highly engaging chatbots,” he says. But there are also novel applications, corresponding to a “chatbot for cancer patients,” where patients upload their history after which share their current symptoms so the bot can learn and offer guidance over time.

It’s price noting that MemGPT isn’t the only one working on this. LangChain might be its best-known competitor and already offers business options. Major modelers also offer tools for creating AI agents, corresponding to the OpenAI Assistant API.

And OpenAI’s latest o1 model could make the need for state fixing a moot point for users. Because it’s a multi-stage model, it essentially needs to take care of some level of state to “think” and fact-check before responding.

But Wooders, Packer, and Myers see some key differences between what Letta offers and what the 800-pound gorilla of the market, OpenAI, does. Letta says it can work with any AI model, and it expects its users to make use of many of them: OpenAI, Anthropic, Mistral, their very own models. OpenAI’s technology currently works only with itself.

More importantly, Letta uses the open-source MemGPT code and firmly sides with the open-source side of the FOSS vs. black-box LLM debate, arguing that open-source is the more sensible choice for AI application developers.

“We’re positioning ourselves as an open alternative to OpenAI,” Packer says. “I think it’s actually very, very hard to build very good AI applications, especially when you’re after something like hallucination, if you can’t see what’s going on under the hood.”