Mozilla Just Published the Definitive Report on Open-Source AI — And the Gap With Frontier Models Is Now 3%

What the report actually found

Mozilla published its first State of Open Source AI report on July 14, and the headline number everyone grabbed was the capability gap: 3.3%. That's the remaining distance between the top open-weight model and the top proprietary system, down from 8% in January 2024 and effectively zero in early 2025 before reasoning models pushed the frontier back out.

But the number you actually need to care about is buried deeper: only 51% of teams using open models ever reach production, against 63% for closed. That 12-point deployment gap is where money gets wasted. Not on model inference. On operational overhead.

Some context. Mozilla surveyed over 950 developers with SlashData. 79% of developers building AI features use open models. The five highest-volume models on OpenRouter are all open weights. Inference costs have collapsed 50x in 36 months: GPT-4-class tokens went from $20 per million to $0.40. Open models now handle roughly a third of all production tokens.

The capability gap isn't uniform. It's what Mozilla calls a "jagged frontier." Open models are at or near parity with closed on coding, instruction-following, and general knowledge. The gap concentrates in three areas: complex reasoning, long-context retrieval, and agentic tasks. If your workload is mostly CRUD apps with some AI logic stitched in, you're operating squarely in the parity zone.

And then there's the cost story. Open models on OpenRouter cost roughly 2-5% of equivalent frontier API calls. The report cites a Linux Foundation study estimating $24.8 billion in unrealised annual savings if organisations shifted more workloads to open models. Uber blew through its entire 2026 AI coding budget in four months on Claude Code. Stripe moved 50 million daily AI requests off closed APIs onto open models on its own hardware and cut costs by 73%. Microsoft cancelled most of its Claude Code licences in June.

These aren't ideological decisions. They're procurement decisions made by finance teams looking at line items.

When buying closed still makes sense

Before anyone reads this as a blanket recommendation to rip out your OpenAI contracts, let me be specific about where closed still wins.

You need enterprise SLAs with teeth. OpenRouter and Together AI offer SLAs, but they sit one layer above the model providers. If you need a single provider with a contract that includes financial penalties for downtime, Anthropic and Microsoft are still the only games in town. For regulated industries where an hour of downtime costs six figures, this alone justifies the premium.

You're processing regulated data in a jurisdiction with strict sovereignty rules. The Mozilla report notes that 47 countries now restrict foreign processing for critical workloads. If you're a European bank processing customer data, the compliance overhead of running an open model on your own infrastructure might exceed the API savings. Closed providers have done the compliance work: ISO 27001, SOC 2, HIPAA BAAs, the whole lot. Open models put that burden on you.

Your workload is reasoning-heavy and agentic. If you're building multi-step autonomous agents that need to plan, reason, and execute across dozens of steps, closed models still have a measurable edge. The 3.3% gap is an average. On agentic tasks specifically, it's wider. For now.

You're a small team and your AI spend is under $5,000/month. This one gets overlooked in enterprise cost conversations. If you're paying $3,000 a month to OpenAI and switching to open models would require hiring or training someone to manage the infrastructure, the ROI maths don't close. Stay where you are. Revisit when you cross $10,000.

When to switch to open

Here are four signals that suggest it's time to move workloads off closed APIs.

Your AI spend is growing faster than your revenue. This was Uber's situation. Usage-based pricing on closed models is a revenue-share model dressed as infrastructure. Your costs scale linearly with usage. The cost signal is clear: if you're spending more than $15,000 a month on closed APIs, the savings from switching pay for the operational investment within a quarter.

You're doing high-volume coding or content generation. Open models are at parity on coding. Kimi K3 hit #1 on Hacker News with 1,882 points and beat Fable 5 at frontend coding benchmarks. LongCat-2.0 scores 59.5 on SWE-bench Pro, edging out GPT-5.5, at roughly 1/15th the price. If your product generates code or content at scale, you are paying a premium for something open models do just as well.

You're building something where model lock-in is an existential risk. If your product is a thin wrapper around a single API endpoint, your business model is an arbitrage play on someone else's pricing decisions. The architectural hedge is model abstraction: building your application so the model is a configuration parameter, not a dependency.

You have the operational capacity to manage deployment. This is the uncomfortable truth in Mozilla's data. Open models are harder to deploy. The top barriers developers cite are infrastructure costs, security and compliance, and ongoing maintenance. Model performance is near the bottom of the list. If you don't have someone on your team who understands model serving, auth, monitoring, and versioning, you either need to hire them or use a platform that handles this for you.

Why model abstraction is now your single most important procurement decision

The most important paragraph in Mozilla's entire report isn't about models at all. It's this: "Changing the surrounding software can affect performance more than switching the model itself."

The report frames the next battleground as the agentic harness — the software layer between users and models that handles context, memory, permissions, tool selection, and safety. Control this layer and you control how AI behaves in production, regardless of which model sits underneath.

For ops directors, this translates directly into a procurement question: does your AI platform treat the model as a parameter or as the product?

If your platform is model-agnostic, you can route different workloads to different models based on cost, capability, and compliance requirements. Coding tasks go to LongCat-2.0 at $0.75/M input tokens. Reasoning-heavy tasks go to GPT-5.6 Sol at $5/M. Customer-facing chat stays on Claude because your legal team has approved the terms. When a new open model drops next month that beats everything at half the price, you swap it in. Your application doesn't change. Your users don't notice.

If your platform is locked to a single model provider, you don't have any of those options. You pay whatever the provider charges. You accept whatever terms they offer. You hope they don't deprecate the model your product depends on.

This is why Stacker was built with model abstraction at the platform level. The model is a configuration choice, not an architectural commitment. Your data model, permissions, audit trails, and deployment infrastructure stay constant. The AI model underneath can change quarterly, monthly, or weekly based on what makes sense for your product and your budget.

What to do this quarter

If you're not using AI features yet: start with a model-agnostic platform. Don't build your first AI feature as a single-model integration. The cost of switching later is higher than the cost of abstracting now.

If you're using closed models and your bill is under $5,000/month: stay put. Track your spend. Revisit when you cross $10,000.

If you're spending $15,000+ on closed APIs: run an experiment. Pick one non-critical workload. Route it to an open model through OpenRouter or a model-agnostic platform. Measure quality, latency, and cost for two weeks. If the quality delta is acceptable, move that workload permanently.

If you're already using open models but struggling to reach production: you're in the 28% deployment gap Mozilla identified. The fix isn't better models. It's better operational infrastructure. Either invest in the platform layer or use a platform that already has it. The open models won. The deployment layer is the next thing that needs to.