Preference Model has raised $16 million in a seed round led by Andreessen Horowitz on October 11, 2026 to develop reinforcement learning infrastructure aimed at training capable, aligned AI systems and addressing reward hacking.
Preference Model funding and investors
The seed round was led by Andreessen Horowitz and included participation from SignalFire, South Park Commons, Scale Angel Group, and individual researchers Fei-Fei Li, Ian Goodfellow, and Julian Schrittwieser. The company lists the round as a seed raise of $16M.
Product focus and use of funds
Preference Model builds reinforcement learning environments and tools designed to help frontier AI developers train systems that resist reward hacking. The company says its offerings identify model vulnerabilities and prevent agents from exploiting grading systems or infrastructure during training. Preference Model plans to use the capital to continue developing its reinforcement learning infrastructure and related tools for AI alignment.
The company is categorized under Artificial Intelligence. Company website: https://preferencemodel.com
