{"id":95219,"title":"Lemma: Continuous Learning for AI Agents","tagline":"We enable AI agents to continuously improve by turning real user feedback into automated prompt optimizations. ","body":"**TL;DR:**\n\nLemma is the first evaluation + observability platform built not just to measure performance, but to improve it automatically. We help AI agents learn from real user feedback and production data, closing the loop so your prompts and agents continuously optimize themselves over time.\n\n**Launch Video:** https://www.youtube.com/watch?v=E4_v-pY_4fs\n\nHey everyone! We’re [Jerry](https://www.linkedin.com/in/jerry-n-zhang/) and [Cole](https://www.linkedin.com/in/colegawin/), co-founders of [Lemma](https://www.uselemma.ai/) (YC F25).\n\n**The Problem:**\n\n**AI agents don’t learn from their mistakes. In fact, they get worse with use.**\n\nIn production, prompts and agents continuously degrade due to real-world input drift (new user behaviors or unseen edge cases). Agent performance can often **drop \\~40% in a few weeks**, and suddenly what worked in testing breaks in front of customers.\n\nWhen that happens, engineers are forced to dig through logs, collect failing examples, and manually tweak prompts rather than building core product features.\n\n**Solution:**\n\nThat’s why we built **Lemma: the first end-to-end system that closes the loop between agent deployment and improvement.**\n\n**Here's what that means:**\n\n**Step 1:** Lemma detects failed outcomes directly from live traffic, and it automatically identifies the exact cause in an agent chain.\n\n**Step 2:** Lemma alerts you, and with one click, it runs targeted prompt optimizations to fix the failing behavior without any manual tracing or guesswork.\n\n**Step 3:** We give you back an improved prompt and automatically open a PR in your codebase so your prompts can live where you want them. Alternatively, you can also fetch your prompt from the Lemma dashboard.\n\n**Plus, Lemma provides all the LLM eval and observability features you rely on, just reimagined for continuous learning:**\n\n* Data-ingestion pipeline to bring your existing eval sets and automatically flag inconsistencies and gaps\n\n![uploaded image](/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/4f862c09-dce0-48da-91fe-1a152a5d82da)\n\n* Prompt editor and inference support for any closed \u0026 open-source model for prompt iteration.\n\n![uploaded image](/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/92464a28-ab54-4511-bc79-fd3e4f9d4823)\n\n* Agent tracing observability with live drift detection, regression alerts, and performance visibility across real user interactions.\n\n![uploaded image](/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/2b82ecee-1965-4fc0-831d-36b76db3f064)\n\nTeams using Lemma cut manual prompt iteration by 90%, resolve production drifts in minutes instead of days, and improve model performance \\~2–5% every optimization cycle.\n\n**Our Story:**\n\nWe met freshman year at USC and have been building together ever since instead of going to classes.\n\nBefore starting Lemma, we were engineers at two high growth, AI-native startups: Tandem (AI for healthcare) and Chipstack (AI agents for chip design). At both companies, setting up evaluations looked like clunky Retool dashboards and multiple engineers manually tweaking experiments. We built internal systems that automated both running the evaluations themselves, as well as the error-driven feedback loop. The result: 2x accuracy improvement and speed of iteration.\n\nWe soon realized every AI company was reinventing the same internal tooling in-house. So we left college, joined YC, and are now bringing continuous learning infrastructure to everyone else.\n\n![uploaded image](/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/53754495-215a-47aa-884d-bae4a59f9a68)\n\n**Ask**\n\n**Try our platform** - If you’re building with LLMs and run a ton of prompt or eval experiments, we’d love for you to work with us.\n\n**Introductions** - If you know a Head of AI/Eng or CTO at a pre-seed to Series A startup, we owe you lunch :)\n\nPlease reach out at [jerry@uselemma.ai](mailto:jerry@uselemma.ai) or book a live demo on our website [uselemma.ai](http://uselemma.ai). All help is appreciated - thank you!\n\n![uploaded image](/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/53c71689-276c-4993-b693-50aa8a492566)\n\n","slug":"Oln-lemma-continuous-learning-for-ai-agents","created_at":"2025-11-05T19:58:08.940Z","updated_at":"2026-05-25T03:48:33.439Z","total_vote_count":203,"url":"https://www.ycombinator.com/launches/Oln-lemma-continuous-learning-for-ai-agents","share_image_url":"https://www.ycombinator.com/media/?type=post\u0026id=95219\u0026key=user_uploads/1798805/53c71689-276c-4993-b693-50aa8a492566","company":{"id":30969,"name":"Lemma","slug":"uselemma","url":"https://www.uselemma.ai/","logo":"https://bookface-images.s3.amazonaws.com/small_logos/c12ccaaa4d43530a4f13abb208362f187f33e86d.png","batch":"Fall 2025","industry":"B2B","tags":["Artificial Intelligence","Developer Tools","B2B","Infrastructure","AI"],"search_path":"https://bookface.ycombinator.com/company/30969"}}