The 17-20x is output token pricing: MiniMax M2.5 Standard at $1.20/M output vs Opus 4.6 at $25/M = ~21x. On input it's $0.30 vs $5.00 = ~17x. Blended (3:1 input:output) works out to roughly 19x.
Curious where you're getting 0.07 and 0.55 — are you looking at M2.5 Lightning pricing or a different provider?
Nobody in the know is paying for Claude anymore, it's so overpriced. It's just the enterprise users.
Well the API is certainly overpriced, no one will pay for that, and that's what these benchmarks use...
But the subscription is still under 20x
MiniMax M2.5 is beating Claude Opus 4.6 and MiniMax is 17x-20x cheaper….why isn’t anyone talking about this?
Sounds like one peculiar benchmark that a particular model could be trained on.
Btw, how do you get 17-20x? Cost seems to be 0.07 to 0.55 (for 4.6, or 8x) and 0.75 (for 4.5 or 11x).
The 17-20x is output token pricing: MiniMax M2.5 Standard at $1.20/M output vs Opus 4.6 at $25/M = ~21x. On input it's $0.30 vs $5.00 = ~17x. Blended (3:1 input:output) works out to roughly 19x. Curious where you're getting 0.07 and 0.55 — are you looking at M2.5 Lightning pricing or a different provider?
I look at "Avg $" (average task cost per instance) from the page you linked to.
Because it's not true? The beating portion, not the cheaper portion.
Why is it not true?
SWE bench is the standard bench to measure an LLMs coding capabilities