Ornith 1.5 Beats Claude Opus 4.8? The Real Numbers
Sharan Sifat13 min read6 viewsOrnith 1.5 proposes its own training tasks, ships under MIT, and is being reported everywhere as beating Claude Opus 4.8. Its own benchmark table says something more careful, and the 800GB footnote matters more than the scores.
On this page
Ornith 1.5 is a family of open-weight models released in August 2026 that generates its own training tasks, writes its own scaffolds for them, and feeds the results back into reinforcement learning. It ships under the MIT licence in three sizes, and it is being reported across the web as beating Claude Opus 4.8.
That last part deserves a closer look. Ornith's own model card describes the flagship as performing "comparably to Claude Opus 4.8," not beating it. Their own published benchmark table shows the model losing two of five comparisons against Opus 4.8, and winning a third by two tenths of a point. Against GLM 5.2 the picture is different and the wins are real.
We are a software agency, not a model lab, so what we care about is whether any of this changes what you should build on. The honest answer is yes, but for reasons that have almost nothing to do with the leaderboard. This article walks through the numbers, the licence, and the 800GB detail that most coverage skipped entirely.
- Ornith 1.5 ships in three sizes under MIT: a 397B MoE, a 35B MoE with 3B active per token, and a 9B dense model.
- Against Claude Opus 4.8 on Ornith's own table it wins three benchmarks and loses two. "Beats" is an overstatement.
- Against GLM 5.2 it wins all five, so that half of the claim holds up.
- The 397B needs roughly 800GB in bf16 and multi-GPU serving. Open weights is not the same as runnable.
- The 35B and 9B are the commercially interesting releases, not the flagship.
What Ornith 1.5 actually is
The three models
Ornith 1.5 comes in three scales, all published on Hugging Face. The flagship is a 397B mixture-of-experts model. Below it sits a 35B MoE that activates only 3B parameters per token, which is what makes it cheap to run relative to its size. At the bottom is a 9B dense model, including a quantized "Mobile" variant aimed at iPhone and Android.
Per the 397B model card, the architecture builds on Qwen3.5 and Gemma 4 with additional continued pretraining, mid-training and post-training. Context is 262,144 tokens, extendable toward a million with YaRN scaling. It is a reasoning model that emits thinking blocks before its final answer, and it serves through vLLM or SGLang behind an OpenAI-compatible endpoint.
GIGAZINE reported the release on 19 August 2026. Quantized builds in GGUF, MLX and NVFP4 formats went up alongside the base weights, which matters more than it sounds — it means people can actually try the smaller models on ordinary hardware from day one.
What "trains itself" means
The self-training claim is the genuinely novel part, and it is more specific than the phrase suggests. Ornith 1.5 is not learning continuously in production from your data. The loop runs during training, in the lab, before the weights you download are frozen.
What it does is remove the humans from curriculum design. Normally somebody assembles a training set of coding tasks. Here the model proposes the tasks itself, then builds the tooling to attempt them, then attempts them, and the whole chain gets rewarded together.
How the self-improvement loop works
Propose, scaffold, solve
The Ornith 1.5 technical post describes three stages that feed each other.
- Task proposal. The model proposes progressively harder tasks that go beyond what it has already solved.
- Scaffold construction. For each task it generates or refines a task-specific scaffold: the instructions, the tools, the decomposition strategy.
- Solution rollout. Conditioned on the task and its scaffold, the policy produces an attempt.
Reward from the rollout is then propagated back across all three stages using GRPO. That is the clever bit. The model is not just rewarded for solving the problem, it is rewarded for having proposed a useful problem and built a workable scaffold for it in the first place.
The reward that keeps it honest
Left alone, a model asked to invent its own homework will invent easy homework. Ornith's answer is a reward that multiplies three terms: validity, frontier difficulty, and novelty.
Frontier difficulty targets a success rate near a set point the team puts at 0.2. Tasks the model solves every time score badly. So do tasks it never solves. The sweet spot is roughly a one-in-five success rate, which is hard enough to be informative and easy enough to produce some successful trajectories to learn from. Novelty is computed as one minus the maximum similarity to previously generated or trained-on tasks, which penalises the model for asking itself the same question again in different words.
A model that grades its own homework needs a reason to make it hard. The difficulty target is doing more work here than the architecture.
Does it actually beat Claude Opus 4.8?
The full benchmark table
Here are Ornith's own published figures for the 397B flagship, reproduced in full rather than selectively. All Ornith results are averaged over five independent runs.
| Benchmark | Ornith 1.5 (397B) | Claude Opus 4.8 | GLM 5.2 | DeepSeek V4 Flash |
|---|---|---|---|---|
| Terminal-Bench 2.1 (Terminus-2) | 86.1 | 85.0 | 81.0 | 82.7 |
| Terminal-Bench 2.1 (Claude Code) | 85.2 | 78.9 | 82.7 | 81.8 |
| SWE-Bench Verified | 86.0 | 85.8 | 83.0 | 81.6 |
| DeepSWE | 56.0 | 59.0 | 46.2 | 54.4 |
| GPQA Diamond | 92.8 | 93.6 | 91.2 | 91.4 |
Read it honestly and the scoreline against Opus 4.8 is three wins and two losses. One of those wins, SWE-Bench Verified at 86.0 against 85.8, is a margin of two tenths of a point on a benchmark of roughly 500 problems. That is one problem. It is not a result you would bet a technology decision on.
The one genuinely decisive win is Terminal-Bench 2.1 under the Claude Code harness, where Ornith posts 85.2 against 78.9. The two losses are not trivial either: DeepSWE by three points, and GPQA Diamond, a graduate-level science reasoning test, by just under a point.
Against GLM 5.2 the story is cleaner. Ornith wins all five comparisons, with the DeepSWE gap being very wide at 56.0 against 46.2. If the headline had been "beats GLM 5.2 and matches Opus 4.8," it would have been accurate.
What the vendor's own wording says
The most telling detail is that Ornith does not make the claim its coverage makes on its behalf. The 397B model card states the model performs "comparably to Claude Opus 4.8." The technical post uses "on par with." Neither says beats.
Every number in that table, including the competitors' scores, was produced by Ornith. Vendor-run evaluations of rivals are normal practice and not evidence of bad faith, but they are not independent verification either. To Ornith's credit the post documents its anti-gaming measures, including removing git history from the repository image and disabling network access during evaluation. Treat the table as a well-documented vendor claim, and wait for third-party leaderboards before treating it as settled.
It is also worth knowing what SWE-Bench Verified is before reading much into a fractional lead on it. Per the SWE-bench project, it is a set of 500 engineer-confirmed solvable problems drawn from real GitHub issues, released in August 2024, where the model must produce a patch that fixes the issue. It is a good benchmark. It is still 500 problems, and single-digit-decimal differences on it are noise.
The catch almost nobody mentions
You probably cannot run the 397B
Here is the line from the model card that changes the practical story: serving the 397B requires multiple GPUs and roughly 800GB in bf16.
- ~800GBbf16 weights for the 397B
- 262Kcontext tokens, extendable with YaRN
- 3Bactive parameters per token on the 35B
- MITlicence on the published weights
Open weights and runnable weights are different things. For the overwhelming majority of businesses, standing up multi-GPU infrastructure to serve an 800GB model costs considerably more than paying for API access to a frontier model, before anyone has written a line of product code. The flagship is a research artifact and a statement of capability. It is not a deployment plan for a company with a normal budget.
The models that actually matter commercially
Which is why the interesting releases are the smaller two, and they get almost no coverage because they do not beat anything famous.
The 35B MoE activates just 3B parameters per token. On Ornith's figures it scores 67.8 on Terminal-Bench 2.1 against 52.5 for the similarly sized Qwen 3.6-35B, and 79.0 on SWE-Bench Verified against 73.4. Against dense models in that class the gap is wider still, with 68.5 on agentic coding versus 43.4 for Gemma 4-31B and 51.7 for Muse Glimmer-30B.
The 9B dense model posts 70.6 on SWE-Bench Verified and 46.2 on Terminal-Bench 2.1, which Ornith notes matches or exceeds considerably larger models. There is a quantized Mobile build for phones.
If you want to evaluate this seriously, start with the 35B MLX or GGUF build on hardware you already own, not the flagship. A model you can actually run and measure against your own tasks tells you more in an afternoon than any leaderboard will. The same logic applies to the small on-device models we covered in our piece on Cactus Needle 2.
Why the MIT licence is the real headline
What MIT actually permits
Ornith 1.5 is published under MIT, which is about as permissive as licences get. You can use it commercially, modify it, fine-tune it on your own data, ship it inside a closed-source product, and owe nothing to anybody. No revenue share, no field-of-use restriction, no monthly-active-user threshold that flips you into a commercial agreement.
That is not the default any more. We wrote about Alibaba reportedly attaching revenue-sharing terms to a Qwen release in our look at open-source AI licensing, and the direction of travel across the industry has been toward more restrictive terms, not fewer. A genuinely MIT-licensed model in this capability class is the part of this release with the longest-lasting consequences.
What it does not solve
A permissive licence removes the legal constraint. It does not remove the operational one. You still have to host it, monitor it, patch it, handle load, and own every failure yourself. Weigh that against an API where somebody else does all of it and upgrades the model underneath you for free.
The cases where self-hosting genuinely wins are narrow and worth naming: data that legally cannot leave your infrastructure, workloads heavy enough that per-token pricing has become the dominant cost line, a need for a frozen model version that will never change under you, or genuine offline operation.
The risk in training on your own output
A model that generates its own training data is copying from itself, and the well-known hazard is gradual degradation: the outputs narrow, the errors compound, and the model gets confidently worse in ways the loop cannot see because the loop is the thing producing the evidence.
Ornith's reward design is clearly built with this in mind. The novelty term exists specifically to stop the model regenerating variations of tasks it has already trained on, and the difficulty target stops it drifting toward problems it finds comfortable. Those are the right countermeasures.
Whether they hold over many more iterations is an open question, and it is the one to watch across future releases rather than the benchmark deltas. If version 2.0 shows the same shape of improvement, the loop works. If gains flatten or the model gets narrower in ways benchmarks do not capture, that will be the more interesting result.
None of this affects you at inference time. The self-improvement loop runs during training, before the weights are published. The model you download does not continue learning from your usage, which is worth being clear about if anyone raises it as a data-privacy concern.
What this means if you are building a product
Three practical conclusions, none of which involve switching anything today.
Do not change your stack over a benchmark margin. A two-tenths-of-a-point lead on a 500-problem benchmark run by the vendor is not a reason to migrate a production system. Benchmarks measure benchmark performance. What matters is how a model behaves on your codebase, your prompts and your failure cases, which you can only find out by running your own evaluation set. That discipline is the same one we argued for in our breakdown of Google's AMIE architecture: build the scorecard before you believe anyone's numbers, including your own.
Do take the small models seriously. A 9B model scoring 70.6 on SWE-Bench Verified, under MIT, with a mobile build, is a real change in what a small team can put inside a product without an API bill attached. That is the sentence in this release with commercial consequences.
Do keep an eye on where licensing goes. The competitive pressure that a capable MIT-licensed model puts on closed pricing is good for everyone building on top. Whether it lasts is the thing to watch.
If you are weighing self-hosted against API for an AI feature, the honest answer usually depends on volume, data sensitivity and how much operational load your team can carry. We work through that trade-off in AI agent development: what to automate versus keep human, and the wider AI Applications cluster covers the surrounding decisions. Building it into a real product is what our SaaS development work covers.
Frequently asked questions
Does Ornith 1.5 really beat Claude Opus 4.8?
Not consistently. On Ornith's own published table the 397B wins three of five comparisons and loses two, including DeepSWE by three points and GPQA Diamond by just under a point. One of the wins is a two-tenths margin on SWE-Bench Verified. Ornith's own model card describes the model as performing "comparably" to Opus 4.8 rather than beating it, which is the more accurate description.
Does it beat GLM 5.2?
On these figures, yes. Ornith 1.5 leads GLM 5.2 across all five reported benchmarks, with a particularly wide gap on DeepSWE at 56.0 against 46.2. That half of the headline claim holds up better than the Opus comparison does.
Can I actually run it myself?
The flagship, realistically no. The model card states the 397B needs multi-GPU serving and roughly 800GB in bf16, which for most businesses costs more than API access to a frontier model. The 35B MoE and the 9B dense model are genuinely runnable, with GGUF and MLX builds available, and those are the ones worth testing.
What does "trains itself" actually mean here?
During training, the model proposes its own tasks, builds scaffolds to attempt them, and produces solution attempts, with reward propagated back across all three stages. It replaces human curriculum design, not human oversight. Importantly, this happens in the lab before release. The weights you download do not keep learning from your usage.
Is the MIT licence genuine?
The published model cards state MIT, which permits commercial use, modification and inclusion in closed-source products with no revenue-sharing obligation. Verify the licence on the specific model card before you commit, since terms on open-weight releases have been shifting across the industry.
Should I switch my product to it?
Not on the basis of these benchmarks. Build an evaluation set from your own tasks and measure the candidates against it, because how a model performs on your codebase is only loosely related to how it performs on public benchmarks. Self-hosting makes sense when data cannot leave your infrastructure, volume has made per-token pricing your dominant cost, or you need a model version that never changes under you.
Weighing an open-weight model against an API for something you are building, and want a straight answer on which actually costs less at your volume? Get a free scope and quote.
Sources
Have a project like this in mind?
Tell us what you're building and we'll map out the scope, timeline and a fixed starting quote — no obligation.
Start your project
