{"id":105382,"title":"Instance: Automated evaluation for robot policies, starting with the success detector","tagline":"Today, evaluating a robot policy means humans have to watch the robot roll out, mark success, and reset the scene – we're automating that.","body":"**TL;DR:** We’re **automating robot evaluation**– the loop where you wait for your robot to roll out, judge its success, and then reset the scene. Today, we’re launching step one: a **success detector** that automatically tells you whether your robot actually completed its task. Book a demo with [founders@instancelabs.ai](mailto:founders@instancelabs.ai) and we’ll visit your office to show you how it works!\n\n\u003chttps://youtu.be/EqxkPffnUKk\u003e\n\n**The problem**\n\nIn Lucy’s robotics research this past year, she **spent way too much time being a babysitter for her robot**. Her project was to get a humanoid to find objects around a room, so her days went like this: start a rollout, watch the robot open all the drawers in the room for 30 minutes, mark success/fail in a spreadsheet, then get up, close all the drawers, and put the objects back where they started. Then do it again, and again, and again. Some days ended with back pain, and for weeks she couldn’t work on anything else because she was stuck doing exactly this.\n\nThat was one robot, in one lab, running a single policy. Industry teams run fleets of robots across dozens of policies, with **multiple people assigned to sit and watch each one**. Every robotics team we’ve talked to has some version of this– and it will only grow as robot foundation models scale.\n\n**Our solution**\n\nInstance is building the **autonomous evaluation rig for robot learning**: your robot rolls out, we judge success, a second robot resets the scene, and the next rollout starts– with no humans in the loop at all.\n\nToday, we’re starting with the success judge. Point us at a task description and your video rollouts, and we return a verdict with detailed subtask captions. We benchmarked our verifier against 10,000+ held-out, human-labeled episodes across 7 robot platforms, and it resulted in **higher accuracy than Claude Opus 4.8, at a fraction of the latency.**\n\n**Try it**\n\nTry it at [demo.instancelabs.ai](http://demo.instancelabs.ai) – drop in a video, or record one from your phone and watch it get verified live.\n\n**Team**\n\nWe’re Claire and Lucy, MIT grads and friends since middle school! \n\nClaire studied math and CS at MIT. At NASA JPL, she built software to simulate planetary atmospheres, and at the MIT Media Lab, she researched sustainable propulsion for spaceflight. She loves working where software meets the physical world.\n\nLucy studied CS at MIT and completed her MEng in MIT's Learning and Intelligent Systems lab. She’s an experienced robotics researcher, and has in the past worked on satellite software at SpaceX, LLM unit testing at Amazon, and brain computer interfaces at Blackrock Neurotech.\n\n**Our ask**\n\nIf you are:\n\n* training robot policies\n* running evals\n* deploying robots in the real world\n* or can intro us to someone who is,\n\nEmail us ([founders@instancelabs.ai](mailto:founders@instancelabs.ai)) to book a demo. We’d love to swing by your office with sweet treats to show you how it works!\n\n![uploaded image](/media/?type=post\u0026id=105382\u0026key=user_uploads/3107325/d03f1074-ae6e-4ab3-acdd-81ff69be93b6)\n\n","slug":"RPi-instance-automated-evaluation-for-robot-policies-starting-with-the-success-detector","created_at":"2026-07-13T20:48:30.605Z","updated_at":"2026-07-22T13:44:07.291Z","total_vote_count":35,"url":"https://www.ycombinator.com/launches/RPi-instance-automated-evaluation-for-robot-policies-starting-with-the-success-detector","share_image_url":"https://www.ycombinator.com/media/?type=post\u0026id=105382\u0026key=user_uploads/3107325/d03f1074-ae6e-4ab3-acdd-81ff69be93b6","company":{"id":31341,"name":"Instance","slug":"instance","url":"https://www.instancelabs.ai/","logo":"https://bookface-images.s3.amazonaws.com/small_logos/6294b12382598c88510ebe8d73568321732a7d00.png","batch":"Summer 2026","industry":"B2B","tags":["Artificial Intelligence","Robotics","Data Labeling"],"search_path":"https://bookface.ycombinator.com/company/31341"}}