Home›Companies›Raycaster

Agentic AI for knowledge work

Raycaster began as an agentic AI platform for knowledge work, serving dozens of publicly traded companies across life sciences, medical devices, energy, and oil and gas. It has since launched a division dedicated to training and evaluating AI systems in these industries, working directly with leading AI labs and enterprises to improve their ability to perform economically valuable work.
Active Founders
Levi Lian
Levi Lian
Founder/CEO
Raised in a four-generation medical family, Levi chose computer science to accelerate research rather than practice it. At Stanford he worked in AI labs on early foundation model applications, then shipped enterprise-ready AI at Solvvy (acquired by Zoom), Intuit and Uber AI. Levi combines deep life-science empathy with proven AI product execution to compress drug-development timelines.
Company Launches
Biopharma Bench 0.1 – Evaluating AI on Autonomous Drug Development
See original launch post

TL;DR: Everyone is trying to use AI for drug discovery (finding molecules), but 90% of actual pharma time and money gets burned in drug development - people digging through messy SharePoint, Veeva, fixing conflicting batch records (often scanned), updating clinical protocols, and answering FDA audit letters with 20 redline versions and siloed email threads with colleagues.

uploaded image

We built Biopharma Bench 0.1: 12 complete, reconstructed drug companies, 71 multi-hour tasks, and 752 ground-truth checks audited by industry veterans to see what happens when you actually put frontier AI into an employee’s seat.

One interesting finding: DeepSeek V4.1 Flash matched GPT-6 Sol on the actual work (57.0% vs 54.3%) at a sixth of the cost ($0.11 vs $0.64 per task). Meanwhile Claude Opus 5, #2 overall at 63.8%, didn't fully pass a single task.

Check out the interactive traces & leaderboard: https://eval.raycaster.ai/benchmarks/biopharma-bench
Read the deep-dive blog post: https://www.raycaster.ai/blog/biopharma-bench
Join the discussion on Hacker News: https://news.ycombinator.com/item?id=49845891
X: https://x.com/dubrovnik07/status/2103543413933064530
LinkedIn: https://lnkd.in/p/g_PD7jYr

Backstory

Hey everyone! We’re the team behind Raycaster (https://raycaster.ai).

We first started building AI agents for regulated industries like biopharma because we see the inevitable wave of knowledge work getting handled by AI systems with human oversight (definitely not fear-mongering, but in a super exciting way!).

Over the last few months, our major focus has been helping frontier AI labs globally hillclimb capabilities in underserved industries such as biopharma, energy, data centers, among others.

Very quickly, we noticed something broken: almost all bio AI benchmarks test multiple-choice questions or molecular physics. Nobody was testing whether an AI agent could actually do the drug development work, open a folder of 50 disorganized PDFs and spreadsheets, find where two lab reports contradict each other, and draft an FDA-defensible document.

So, we spent the last few months working directly with former pharma directors and regulatory specialists to build an evaluation that tests what real work actually looks like.


The Problem: The "Fluency Trap"

It takes 8–10 years and over $1B to get a drug to patients. Most of that delay is paperwork and coordination mistakes:

  • Protocol Drift: ~75% of clinical trials have to amend their protocols mid-flight. Each amendment costs hundreds of thousands of dollars and months of lost time.
  • Messy Records: A tiny discrepancy between a stability test memo from last Tuesday and an executed batch record can trigger FDA warning letters (Form 483s) or clinical holds.

When people play with LLMs for this, they fall into what we call the fluency trap: Frontier models write seemingly true prose. If you just read the output, the AI sounds like a 20-year VP of Regulatory Affairs who knows everything.

But when you actually grade the facts against ground truth, the model cited an outdated 2023 draft instead of the approved amendment, or hallucinated a purity threshold. In biopharma, being 95% right with one made-up number is an instant rejection.

uploaded image

How Biopharma Bench Works

We didn’t give the models clean, cherry-picked prompts. We dropped them into messy environments:

  • 12 Companies: Real reconstructed company drives packed with thousands of messy files and software states - unredacted study reports, supplier quotes, assay validations, and random internal memos.
  • 71 Long-Horizon Tasks: Multi-hour workflows across Preclinical, CMC/Manufacturing, Clinical Ops, and Regulatory Affairs (like investigating an out-of-spec stability result or drafting a protocol amendment).
  • 752 Ground-Truth Checks: Graded against falsifiable rubrics created using real documents along with former pharma directors.

uploaded image


What Shocked Us from the Runs

We ran frontier reasoning models across multiple popular harnesses (Codex, Pi, Claude Code). Here are the 3 biggest takeaways:

  1. DeepSeek matched GPT-6 Sol at 1/6 the cost. 
  2. Almost nobody finishes a full task cleanly. Top systems get 45–70% of checks, but full passes are rare: GPT-6 Astra 8/71, Grok 4.6 2/71, Kimi K3 and GPT-5.6 Sol 1 each. Claude Opus 5, DeepSeek, GPT-6 Sol, GLM, and Gemini got 0.
  3. The writing is fine; the sourcing isn't. In one public task, Claude Opus 5 did everything right on the surface: it navigated the folders, read scanned protocols page by page, and wrote a clean brief. But a secondary report claimed a "protocol-defined 30-minute window" that doesn't exist in the controlling protocol. Opus repeated it as fact in the brief for the inspector. An experienced engineer would have flagged it.

See the Traces

uploaded image


How We Can Help

  • Naturally as a data / research lab, we have tons of insights into benchmark design - we recently became a proud sponsor of evalscience.org, a 501(c)(3) nonprofit focused on scientific agent evaluation methods and independent benchmark verification. If there’s interest, happy to share more regularly on BookFace
  • We work with / fund academics and researchers and have landed joint papers in NeurIPS, NAACL and all the big conferences. Bring your idea to us and see what can come out of it
  • Finally, if you’re a neo-lab reading this, we have lots of ready to license data, environments and evals on some of the most interesting yet underserved industries 😄 , talk to us at team@raycaster.ai
Previous Launches
AI workspace for high-stakes documents, not just code.
Jobs at Raycaster
New York, NY, US / Remote
$40 - $130 / hourly
3+ years
New York, NY, US / Remote
$40 - $130 / hourly
3+ years
Raycaster
Founded:2025
Batch:Fall 2024
Team Size:6
Status:
Active
Location:New York City, NY
Primary Partner:Diana Hu