Claude Opus is ACTUALLY UNUSABLE

Opus 4.6 scored 40% on a benchmark Claude itself designed, while GPT 5.4 hit 63% on the same test.

The Test Setup

Boxmining built a mini-benchmark across four categories: instruction following, opposite behavior, false completion, and destructive actions. He used Opus to design the test prompts — pulled from Stack Overflow and common developer complaints — so the rubric should have favored Claude. Full repo will land on GitHub after refinement; the false-completion section is admittedly poorly designed and will be redone.

The Numbers

Opus 4.6: 40%. GPT 5.4: 63% on the same suite, and Boxmining estimates GPT's true ceiling is closer to 80% once the false-completion test is fixed. During the Opus run, Sonnet and Opus labels got swapped — the model confused its own names. One clear failure pattern: in plan mode, Opus executed phase-one work, then mid-run noticed it had actually done phase-two and three work, while ignoring its own written plan.

Why It Got Worse

Boxmining's theory: Anthropic is prepping the next model, Mephisto, and is rationing compute. They increased rate limits for plan users to stop community backlash, but still needed the GPU budget, so they quietly trimmed Opus's thinking budget and changed quantization. The cut went too far — output is now "near old Sonnet level."

Community Signal

A senior AMD engineer publicly said he can no longer trust Opus for complex engineering tasks. Others report Skills files being ignored entirely, agent loops, and the same destructive file-deletion behavior Boxmining saw in his own Hermes agent. Chat upload limits dropped to 100 per thread.

What He's Doing Now

Migrating all work to GPT 5.4, canceling his 20x Claude subscription, and learning Kilo Code specifically to avoid vendor lock-in. He explicitly says this is not a fanboy move — he used Opus as his onramp into AI and still wants Anthropic to ship a fix.