{"id":111310,"title":"Agnost AI: Turn Your Agent Traces Into a Faster, Cheaper, \u0026 more Accurate Custom Model","tagline":"We use your agent’s production traces to train a specialized model that performs better, responds faster, and costs less.","body":"Your agent logs aren’t just for debugging. They’re your training data.\\\nHey YC! We’re Shubham and Parth, childhood friends and founders of Agnost AI.\n\n**TL;DR**: Agnost AI uses your agent’s production traces to train a model specialized for your exact workload. Our first custom model for our customer delivered them a:\n\n* 22.9% higher task success,\n* 90.2% lower median latency, and\n* 94.5% lower cost than Opus 4.8.\n\nDemo: \u003chttps://youtu.be/R3NyWNYXHu4\u003e\n\n### What we found\n\nA few weeks ago, we launched Agnost AI as product analytics for AI agents. We read conversations between users and agents to uncover failures, behavior drift, hallucinated links, frustration, and churn signals.\n\nWe expected to find bugs. Surprisingly, we also found a lot of infrastructure waste. Many customer-facing agents use frontier models for narrow, repetitive jobs:\n\n* Route a request to the correct tool\n* Have a narrow system prompt with the given inputs\n* Retrieve the relevant data from internal tool calls\n* Parse \u0026amp; format the final response\n\nThe underlying frontier model can write code, solve advanced mathematics, understand biology, and reason across hundreds of domains. **Your agent uses a small, repeatable slice of that capability while paying frontier-model prices for every request.**\n\n### We trained a specialist\n\nWe used historical production traces to train our first workload-specific model for an early customer: **agnost-\u0026lt;redacted\u003e-0.1**\n\nWe evaluated it against Opus 4.8 on 780 held-out customer traces for an Identifier Extraction Agent. The results:\n\n* Task success: 71.5% → 87.9%, 22.9% relative improvement\n* Median latency: 4.80s → 0.47s, 90.2% lower\n* Tail latency: 11.20s → 1.35s, 87.9% lower\n* Cost per 1,000 completed tasks: $42.00 → $2.30, 94.5% lower\n\nThis is one model for one workload, not a claim that specialist models should replace frontier models everywhere.\n\nFrontier models remain the right choice for open-ended work. But once an agent’s production job becomes predictable, **paying for general intelligence on every request stops making sense**.\n\n### How it works\n\n1. Send your existing traces to Agnost through OpenTelemetry or our SDK.\n2. We identify the recurring workflows your agent performs.\n3. We create an eval set from held-out production traces.\n4. We train a specialist model on the historical workload.\n5. We benchmark it against your current model on success, latency, and cost.\n\nYou only consider switching when the specialist wins on your own workload. Your logs go from debugging data to evaluation data and finally to training data recursively.\n\n### Who we are\n\nWe’re childhood friends, eight years and counting.\n\n* Shubham: Youngest engineer on Cisco’s analytics team, first hire at Formbricks, and contributor to Bitcoin, OWASP, and the Linux Foundation.\n* Parth: IIT Madras CS, ranked 159 out of 1.5 million. Built Spark pipelines at Microsoft and led GPU technology at Infurnia.\n\n### Our ask\n\nIf you have meaningful daily traffic, send us an anonymized trace export. We’ll analyze the workload, build a held-out evaluation set, and tell you whether a specialist model is worth training. If it is, we’ll benchmark it against your current frontier model.\n\nEmail me at [shubham@agnost.ai](mailto:shubham@agnost.ai) or book a call at [call.agnost.ai](http://call.agnost.ai)\n\nMore about us at: [agnost.ai](https://agnost.ai)\n\n![uploaded image](/media/?type=post\u0026id=111310\u0026key=user_uploads/1987056/6599ed29-ddf9-4d02-9232-425af1535be7)\n\n","slug":"SxK-agnost-ai-turn-your-agent-traces-into-a-faster-cheaper-more-accurate-custom-model","created_at":"2026-08-20T16:00:00.252Z","updated_at":"2026-09-19T09:44:33.128Z","total_vote_count":96,"url":"https://www.ycombinator.com/launches/SxK-agnost-ai-turn-your-agent-traces-into-a-faster-cheaper-more-accurate-custom-model","share_image_url":"https://www.ycombinator.com/media/?type=post\u0026id=111310\u0026key=user_uploads/1987056/6599ed29-ddf9-4d02-9232-425af1535be7","company":{"id":33019,"name":"Agnost AI","slug":"agnost-ai","url":"https://agnost.ai","logo":"https://bookface-images.s3.amazonaws.com/small_logos/e1595f92ea75b1f367addedb5703852b1c7efded.png","batch":"Summer 2026","industry":"B2B","tags":["Artificial Intelligence","Developer Tools","Machine Learning","Analytics","Monitoring"],"search_path":"https://bookface.ycombinator.com/company/33019"}}