OpenAI and Anthropic costs push Harvey toward cheaper open models
The $15.6 billion legal startup saw gross margins fall from about 50% to -50% after AI agent usage surged, Bloomberg reported.
Harvey is shifting toward open-weight AI models after a surge in customer use sent its gross margins from about 50% at the start of 2026 to -50% by June, according to a person familiar with the matter.
The legal technology startup, valued at $15.6 billion, built its business around models from OpenAI and Anthropic. After a March update to its AI agents, usage increased sharply and pushed up the cost of running the service.
Harvey released its own model in August using Kimi K3 from China-based Moonshot AI. The company said the model can perform close to Anthropic’s best offerings at a fraction of the cost, while Harvey’s gross margins have since turned positive, according to people familiar with the effort.
Open-weight models publish their parameters for developers to download and modify. That gives companies more control over their technology and lets them train models with their own data instead of paying solely for proprietary model access.
Other startups are pursuing similar strategies. Healthtech company Abridge is building a custom foundation model for clinical settings, AI support startup Decagon routes 80% of queries through its own models, and fintech companies including Ramp and Rogo are exploring in-house training.
AI, tech, and the markets they move—in one daily briefing.
Daily. Free. Join 34,000+ readers across crypto, finance, and policy.
Ramp co-CEO Karim Atiyeh said the company, which raised $750 million in June, is considering model-building because open-weight systems have improved. He said the economics made little sense a year ago but are becoming more compelling as compute costs rise.
Investors and executives also warn that custom models require specialized talent, upfront spending and large amounts of proprietary data. Some startups may still find it cheaper to use closed models, especially when traffic is low.
Harvey president and co-founder Gabe Pereyra said the company’s AI token use increased twentyfold this year, a level he described as unsustainable if the startup continued to rely on higher-cost models from OpenAI and Anthropic. Harvey still expects to use those systems for its most demanding tasks.