Continuous optimization for AI agents
Hey everyone! 👋
We're Macy, Faiz and Niyaz - co-founders of Papaya
TL;DR
Papaya is the optimization engine for AI agents. Connect your production agent and Papaya watches every action, finds what's hurting quality, cost, and latency, and gives you the fix. Once a fix is in place, Papaya makes sure your agent doesn't regress. Teams running Papaya are seeing 20%+ improvements to quality, latency, and cost.
Ask: If you run an agent in production, connect it to Papaya. Grab a time to get started here.
The Problem
We’ve been building agentic systems for years (6 years to be exact…before it was cool), including really large ones that helped run major workflows inside Amazon. We know firsthand that the hard part isn’t getting an agent to work, it’s keeping it working at a high quality bar. We consistently saw that every model update and every prompt change shifted behavior in ways nobody caught until something downstream looked wrong. The only way to find out what changed was to pull traces and read them by hand.
Most companies shipping agents today are focused on the next feature for customers and they don't have time to fully maintain what's already been released. When we built our own agent as a three-person team, we hit the same wall. We needed to make sure our agent had high quality answers, was fast for our customers, and as low cost as possible, but we had to look at everything by hand. Every hour we spent examining traces was an hour we couldn’t spend on new product development. That's why we built Papaya.
How Papaya works 🔧
At Papaya, our goal is to do the hard AI engineering work for you, staying up to date on the latest and greatest research so you can take advantage of this knowledge through optimizations. Simply stream your production traces through a lightweight SDK. From there, Papaya:
- Finds your use cases: groups every run by the job the agent is doing, so you're looking at "refund requests," not "run 4812."
- Runs 200+ analyses: checks every use case against 200+ research-backed failure patterns across quality, cost, and latency.
- Ranks the fixes by impact: every recommendation comes with an estimated effect on quality, latency, and cost. Approve the ones you want directly in Papaya and Papaya opens the PR.
- Watches for regression: once a fix ships, Papaya keeps checking that the improvement holds.
Our Ask 🙏
- Run an agent in production? Connect it to Papaya. Grab a time with us here.
- Know any companies spending tons of time maintaining and fixing their agents? Intro us at macy@papaya.fyi.
- Building agents and want a second set of eyes? Ping us. We are happy to help however we can!
The Team
We've spent over 10 years building large scale agentic systems at Amazon, and we've scaled other YC startups from $0 to $10M ARR through acquisition. We started out building an analytics agent. The harness we built to optimize our own agent became the center of every prospect conversation, so we pivoted to it. Meet our team below :)
