{"id":82145,"title":"Parea AI - Aligned \u0026 reliable LLM evaluations","tagline":"Automatically create LLM-based evals aligned with human judgement","body":"**tl;dr:** [Parea AI](https://www.parea.ai/) automates the creation of evals for your AI products. We achieve this by bootstrapping an evaluation function with human annotations. Allowing you to automagically turn “vibe checks” into scalable and reliable evaluations aligned with human judgment.\n\n# **😩 The Problem**\n\n![uploaded image](/media/?type=post\u0026id=82145\u0026key=user_uploads/501039/78024ed1-9aa2-4be8-a022-359d82b9ac59)\n\nEvaluating free-form text is often only possible by humans reviewing outputs or using LLMs to evaluate them. The former is laborious, slow, and expensive, while the latter often fails to evaluate the outputs correctly. For LLM evaluations to work properly, one needs to prompt engineer them; i.e., they require their own optimization process.\n\n# **🚀 The Solution**\n\n![uploaded image](/media/?type=post\u0026id=82145\u0026key=user_uploads/501039/b7c1f4c7-b654-40e6-9e1d-84ab93f4ad5f)\n\nThe best [LLM evals](https://joschkabraun.com) are adapted to your particular business use case \u0026 data. We've developed a method for uploading human annotations (via CSV or using our [Annotation Queue](https://docs.parea.ai/manual-review/queue)) and bootstrapping an evaluation to mimic the annotations. To create a human-aligned eval, you need as few as 20 sample annotations. Using your new LLM eval is as easy as copying the code into your codebase or using it directly via Parea's API. Check out [our docs](https://docs.parea.ai/manual-review/bootstrapped-eval) to see the complete workflow.\n\n# **🙏 Our Ask**\n\n* **Get started** on [**our free tier,**](https://app.parea.ai/sign-up) or [**book a chat**](https://calendly.com/parea-ai/chat) to discuss how we can help you evaluate your AI app!\n* **Support our** [**launch** **tweet**](https://x.com/JoschkaBraun/status/1812884216444510233) and follow us on [**LinkedIn**](https://www.linkedin.com/company/parea-ai/) and [**Twitter**](https://x.com/PareaAI)\n* **Share Parea** **AI** with anyone you know who is facing challenges [evaluating AI \u0026 LLM systems](https://joschkabraun.com).","slug":"LMv-parea-ai-aligned-reliable-llm-evaluations","created_at":"2024-07-15T16:47:45.697Z","updated_at":"2026-05-25T05:27:15.572Z","total_vote_count":8,"url":"https://www.ycombinator.com/launches/LMv-parea-ai-aligned-reliable-llm-evaluations","share_image_url":"https://www.ycombinator.com/media/?type=post\u0026id=82145\u0026key=user_uploads/501039/78024ed1-9aa2-4be8-a022-359d82b9ac59","company":{"id":28846,"name":"Parea","slug":"parea","url":"https://www.parea.ai","logo":"https://bookface-images.s3.amazonaws.com/small_logos/32972e7b6074f606373a477c496dcde4a5e81219.png","batch":"Summer 2023","industry":"B2B","tags":["Developer Tools","Generative AI","SaaS","DevOps","Monitoring"],"search_path":"https://bookface.ycombinator.com/company/28846"}}