A startup referred to as Letta has simply emerged from stealth with tech that helps AI fashions keep in mind customers and conversations. Created in UC Berkeley’s famed Labs startup manufacturing unit, it additionally introduced $10 million in seed cash led by Felicis’ Astasia Myers, at a $70 million post-money valuation.
Letta can also be backed by a who’s who of angel traders in AI, like Google’s Jeff Dean, Hugging Face’s Clem Delangue, Runway’s Cristóbal Valenzuela, and Anyscale’s Robert Nishihara, amongst others.
Based by Berkeley PhD college students Sarah Wooders and Charles Packer, it is a extremely anticipated AI startup launch. That’s as a result of it’s a toddler of Berkeley’s Sky Computing Lab and is the business entity of the favored MemGPT open supply challenge.
Berkeley’s Sky Computing Lab, led by acclaimed professor and Databricks co-founder Ion Stoica, is the descendent of RISELab and AMPLab, which spawned such corporations as Anyscale, Databricks, and SiFive. Sky Lab, particularly, birthed quite a few well-liked open supply giant language mannequin (LLM) initiatives just like the Gorilla LLM, vLLM, and the LLM structured language SGLang.
“A ton of initiatives in a short time, inside a yr’s time-frame, got here out of the lab. Simply individuals sitting subsequent to us,” described Wooders. “So it was type of an unbelievable time.”
MemGPT is one such challenge and is such a scorching commodity that it really went viral earlier than it even launched.
“Somebody scooped us,” Packer advised TechCrunch. The founders had posted a whitepaper on Thursday, October 12, 2023, and deliberate to launch a extra in-depth paper and the code to GitHub the next Monday. Some random particular person discovered the paper, posted it to Hacker Information on Sunday, and it “went viral on Hacker Information earlier than we had an opportunity to correctly launch the code, launch the paper, or, like, do a tweet thread or something like that,” he stated.
The explanation for the joy was that MemGPT mitigates a pernicious drawback for LLMs: Of their native kind, fashions like ChatGPT are stateless, that means they don’t retailer historic knowledge in long-term reminiscence. That is problematic for AI apps that rely on attending to know and be taught from a person over time — every part from buyer help bots to healthcare symptom-tracking apps. MemGPT manages knowledge and reminiscence in order that AI brokers and chatbots can keep in mind earlier customers and conversations.
The submit on the paper stayed atop Hacker Information, the favored website for programmers run by Y Combinator, for 48 hours, Packer recounted. So he spent his weekend and the following few days answering questions on the positioning whereas making an attempt to get the code able to be launched. As soon as the challenge was out there on GitHub, a hyperlink to it went viral on Hacker Information, once more. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K forks on GitHub occurred shortly.
VC Felicis’ Myers found Wooders and Packer by studying about MemGPT, too, and instantly acknowledged the tech’s business potentialities.
“I noticed the paper when it was launched,” she advised TechCrunch, and she or he promptly reached out to the founders. “We had an funding theme round AI agent infrastructure and appreciated {that a} actually necessary part of that was the information and reminiscence administration to make these conversational chat bots and AI brokers efficient.”
The founders nonetheless just about traipsed round Sand Hill Street doing Zoom calls with VCs earlier than going with the one which cherished them first.
In the meantime, Stoica brokered introductions to Dean, Nishihara and different big-name Silicon Valley angels. “Numerous the professors at Berkeley, simply as a consequence of being at Berkeley, are very properly linked,” Packer recalled about how straightforward the angel investor course of was. “They’ve their eye on initiatives out of this lab which might be going to be commercialized.”
Competitors and the specter of OpenAI o1
Whereas MemGPT is already out within the wild and getting used, Letta’s business variant, Letta Cloud, shouldn’t be but open for enterprise. As of Monday, Letta is accepting requests for beta customers. It is going to provide a hosted agent service that enables builders to deploy and run stateful brokers within the cloud, accessible by way of REST APIs, a programming interface that may keep state. Letta Cloud will retailer the long-term knowledge essential to take action. Letta may also provide developer instruments for constructing AI brokers.
With MemGPT, Wooders sees a big span of makes use of. “I believe the primary use case that we see is mainly, extremely customized, very partaking chatbots,” she says. However there are additionally cutting-edge makes use of like “a chatbot for most cancers sufferers” the place sufferers add their historical past after which share ongoing signs so the bot can be taught and provide steering over time.
Value noting that MemGPT isn’t alone in engaged on this. LangChain might be its greatest identified competitor, and it already affords business choices. The largest mannequin makers additionally provide AI agent-making instruments as properly, like OpenAI’s Assistants API.
And OpenAI’s new o1 mannequin might make the necessity to repair state a moot level for its customers. As it’s a multistep mannequin, it essentially should keep state to a point in an effort to “suppose” and reality examine earlier than it replies.
However Wooders, Packer, and Myers see a couple of key variations to what Letta is providing versus what 800-pound market gorilla OpenAI is doing. Letta claims it can work with any AI mannequin and expects its customers to make use of lots of them: OpenAI, Anthropic, Minstrel, their very own homegrown fashions. OpenAI’s tech presently solely works with itself.
Extra importantly, Letta is utilizing open supply MemGPT and leaping firmly into the open supply aspect of the FOSS vs. black field LLM debate, saying open supply is a more sensible choice for AI software programmers.
“We’re positioning ourselves because the open different to OpenAI,” Packer says. “I believe it’s really very, very exhausting to construct excellent AI functions, particularly while you care about, like hallucination, in the event you can’t see what’s happening underneath the hood.”