{"id":89925,"title":"Raindrop - Sentry for AI Products","tagline":" AI engineers use Raindrop to get alerts about hidden issues and wins in their AI products.","body":"[https://youtu.be/IS7akU_3AnI](https://youtu.be/IS7akU_3AnI?feature=shared)\n\nAl products fail constantly—in ways both hilarious and terrifying.\n\nMeet **Raindrop**: the first Sentry-like monitoring platform for Al products.\n\n**The Problem**\n\nRegular software throws exceptions. But Al products fail silently.\n\nThese issues can be serious:\n\n* [Figma had to roll back their AI design product when it copied Apple’s designs, and they haven’t relaunched since](https://techcrunch.com/2024/07/02/figma-disables-its-ai-design-feature-that-appeared-to-be-ripping-off-apples-weather-app/?guccounter=1\u0026guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8\u0026guce_referrer_sig=AQAAAM5aRTBjXhPXfaoMBpoQxL9si-7R3qfaTR4r-n_mdzKu31uB6rUYGjE31z0H4aHVY1LlchyZ7r76Pufga45IMGKwFz0IxZ7eo-g2z1sYsta94AVsSj5CYjf6NKRd3m43VgSGLrG_IfdqMKtlFDuBvimRfLyF_aqb19vPgl-8B7eD)\n* [Air Canada got sued — and lost — when their chatbot incorrectly offered a customer a refund](https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/)\n* [Virgin Money came under fire when their chatbot scolded customers for using the company name, ‘Virgin’](https://www.latintimes.com/virgin-money-forced-apologize-after-chatbot-threatens-cut-off-bank-customers-who-use-word-573885#:\\~:text=Fintech%20commentator%20David%20Birch%20shared,reported%20by%20the%20Financial%20Times)\n\nDon’t make the same mistake.\n\nThe current status quo is sifting through millions of logs and trying debug flaky evals. Evals are straightforward - like unit tests, they confirm your model got specific test cases right. But in the real world AI chatbots and agents encounter millions of unpredictable actions. AI companies need powerful monitoring to discover production issues so they can create reliable AI products that don’t just pass tests, but genuinely resonate with users.\n\n**Solution**\n\nRaindrop sends you alerts when your AI misbehaves and links straight to the events, so you can dig into the conversations or traces, understand the root cause, and fix it, fast.\n\nDaily Alerts Include\n\n* **Issues**: detects issues like Assistant Forgetting, Laziness, Task Failure, User Frustration, and more depending on the type of app\n* **Wins**: surfaces what your product does well, so you can double down on those experiences and create evals\n\nThe Pro tier lets you go even further with:\n\n* **Custom Issues / Topics**: define and track any issue or topic\n* **Topic Clustering**: clusters data in real time to find your AI product’s most popular use cases, and what use cases have the most issues\n* **Signals**: finds patterns in explicit signals like thumbs up / thumbs down\n* **Deep Research:** deep research for your production data, letting you use natural language to search for any kind of behavior\n* **Traces**: track every step of your AI call\n* **Edge PII Redaction**: intelligently strip PII from any user and model messages\n* **Dataset Creation**: create custom datasets out of any set of events\n\nCompanies like [Clay.com](http://Clay.com), Tolans, Websim, and more have been using Raindrop to improve their AI products. They’ve been able to quickly iterate on fixes, see how issue incidence rates decrease in production, and confidently ship changes.\n\n“Raindrop has been invaluable as we've been growing quickly. It's critical for us to keep issue incidence below an acceptable threshold and become aware of any spikes. **It's like if we see an iOS crash report in Sentry, but for our AI capabilities.”** -Evan Goldschmidt, CTO Tolans\n\nLearn more about how Tolan uses Dawn [here](https://www.raindrop.ai/case-studies/tolans), and check out case studies with [New Computer](https://www.raindrop.ai/case-studies/new-computer) and [Unstuck AI](https://www.raindrop.ai/case-studies/unstuck).\n\nYou can get started [here](https://www.raindrop.ai), excited to hear what you think :)\n\n![uploaded image](/media/?type=post\u0026id=89925\u0026key=user_uploads/309605/d77c5ea9-2b73-4455-96a6-db02c319651c)\n\n","slug":"NOP-raindrop-sentry-for-ai-products","created_at":"2025-04-28T20:25:57.382Z","updated_at":"2026-05-25T00:24:04.962Z","total_vote_count":11,"url":"https://www.ycombinator.com/launches/NOP-raindrop-sentry-for-ai-products","share_image_url":"https://www.ycombinator.com/media/?type=post\u0026id=89925\u0026key=user_uploads/309605/d77c5ea9-2b73-4455-96a6-db02c319651c","company":{"id":29401,"name":"Raindrop","slug":"raindrop","url":"https://www.raindrop.ai","logo":"https://bookface-images.s3.amazonaws.com/small_logos/53f3136371147d755b0a7f4891db765c11734499.png","batch":"Winter 2024","industry":"B2B","tags":["Artificial Intelligence","B2B","Monitoring","AI"],"search_path":"https://bookface.ycombinator.com/company/29401"}}