Olam Labs: Evaluating models for behavior and performance through simulated games

Evaluating models for behavior and performance through simulated games

Evaluating models for social behavior, safety, and performance through multi-agent simulated games. We use popular games like Catan, Risk, or Poker, and make humans come play them against talking AI opponents, but on the backend we're actually running a multi-agent environment used to evaluate the models for skill and behavior over large sample sizes. You can play in Social Arena and view our preliminary benchmarks today at https://olamlabs.ai/

Active Founders

Om Buddhdev

Co-Founder

I want to build rly cool simulations At 11, I was posting YouTube videos about building PCs and hardware projects At 12, I was one of the wealthiest players on Minecraft's largest economy sim server At 13 & 14 I was making Roblox games played millions of times At 15 & 16 I was competing in Fortnite, peaking at top 100 NA. At 17 & 18, I was viral on TikTok with 195k followers. At 19 & 20, I was 8x radiant and peaked top 25 in Valorant At 21, I was a Staff Engineer at AI Dungeon / Latitude

Om Buddhdev

Co-Founder

I want to build rly cool simulations At 11, I was posting YouTube videos about building PCs and hardware projects At 12, I was one of the wealthiest players on Minecraft's largest economy sim server At 13 & 14 I was making Roblox games played millions of times At 15 & 16 I was competing in Fortnite, peaking at top 100 NA. At 17 & 18, I was viral on TikTok with 195k followers. At 19 & 20, I was 8x radiant and peaked top 25 in Valorant At 21, I was a Staff Engineer at AI Dungeon / Latitude

shreshth sharma

Founder

currently building simulated envs to create better evals @ olam labs, previously: was doing a cs and business double degree at uwaterloo (dropped out for yc) building reinforcement learning systems and doing data science @ RBC i was briefly top 50 on fifa mobile

shreshth sharma

Founder

currently building simulated envs to create better evals @ olam labs, previously: was doing a cs and business double degree at uwaterloo (dropped out for yc) building reinforcement learning systems and doing data science @ RBC i was briefly top 50 on fifa mobile

YC Photos

Hear from the founders

How did your company get started? (i.e., How did the founders meet? How did you come up with the idea? How did you decide to be a founder?)

Om and Shreshth have always had a deep love for game design since middle school, in their early years making Roblox games with millions of plays and running a charity for their middle school.

Om was among the best Valorant players in the world before becoming a Product and then Staff Engineer at AI Dungeon, working on the frontier of AI gaming.

They convened and fell in love with the idea of creating simulated worlds to evaluate AI agents in, and the qualitative behaviors that they saw in early demos from multi-agent interaction.

They decided to pursue it and applied to YC, and have been in pursuit of a future where we can understand and measure the qualitative traits of the everyday models that we work with.

More on Shreshth: shresh.ca https://x.com/shresh
More on Om: sensho.xyz https://x.com/sensho