91 comments

  • apatheticonion an hour ago ago

    I love the Chinese models.

    I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks.

    DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform).

    I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up.

    It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage.

    OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.

    • arikrahman 35 minutes ago ago

      The best part is using harnesses like reasonix or whale make cache hit at a rate close to 98%, making requests converge to practically free. And that's with unsubsidized American providers like cloudflare or Digital Ocean.

      • mlazos 19 minutes ago ago

        How can you be hitting cache on what I think are novel LLM prompts …

        • oceansweep 4 minutes ago ago

          Not them but my understanding is that the harness will send a simple 'heartbeat' message to keep the cache 'warm', (see prefix caching: https://handbook.modular.com/inference-optimization/prefix-c... ) which can then be edited/changed, which does cause the user to incur a fee, but its much less than the amount they'd pay on a no-cache hit request.

        • verdverm a minute ago ago

          multi turn sessions, they are typically in the high 90% hit rate across all providers without doing much of anything

    • skeledrew 14 minutes ago ago

      Prepaid means someone can't sign up, use a bunch of inference, then cancel their card and disappear into the sunset. VPS is a more long term investment where it's harder to switch and it doesn't cost the provider much if a few users jump out without paying for a month.

    • copperx 43 minutes ago ago

      What don't you like about the service, besides the mark up?

      • apatheticonion 14 minutes ago ago

        For me, the only utility OpenRouter gives me is billing consolidation - I don't really need the routing capabilities because I use Bifrost for that.

        As a router, it's not very feature rich. For example I restricted the available models to the ones I want to use however the `/models` endpoint still lists all the models, making my LLM client list the 200+ models available on the service (even though they will throw an error if I try to use them).

        With Bifrost, I can also create model aliases with custom configuration - for example I can create a model alias `deepseek-v4-flash-nothink` which disables thinking. I can create `deepseek-v4-flash-caveman` which injects the caveman skill (to save tokens) etc.

        Plus I can contribute to Bifrost, which I can't do with OpenRouter.

    • ronsor 42 minutes ago ago

      Have you looked into OpenCode Zen or OpenCode Go?

      https://opencode.ai/zen

  • skybrian a minute ago ago

    [delayed]

  • culi an hour ago ago

    A third the cost, open source, and won't refuse every other request because of some vague possible connection to cybersecurity concerns.

    • dgellow an hour ago ago

      My questions about strawberries got blocked as too dangerous! I’m not joking

      • shikon7 39 minutes ago ago

        Strawberries are a well known weakness of LLMs, as they have a hard time to count the numbers of "r"s in them. Maybe that's why, because they fear that weakness could be exploited somehow.

        • skeledrew 33 minutes ago ago

          Probably need to be taught by someone of Latino origin. Learning to roll them "r"s could help.

          • culi 17 minutes ago ago

            2/3rds of all languages use rolled r's

            • dougSF70 7 minutes ago ago

              2/3rrrrds of all languages use rolled r's

      • teaearlgraycold 19 minutes ago ago

        I asked about Tiananmen Square and it said it knew nothing about it. A real life “Doesn’t look like anything to me” moment.

    • villish 44 minutes ago ago

      From the blog post moonshot refers to it as open source but only mentions releasing the weights.

      • culi 23 minutes ago ago

        So far all the Kimi models have open sourced their code, their weights, and published technical reports explaining their training methodology. I expect the Kimi K3 Technical Report will come out July 27 and they usually publish the code and the weights alongside that.

      • charcircuit 33 minutes ago ago

        The weights are the "source" of a model.

        • tadfisher 31 minutes ago ago

          If weights are the source for models then ELF binaries are the source for software.

          • RobMurray 13 minutes ago ago

            clearly not true. the weights are the preferred form for making modifications. Do you really think people should be downloading hundreds of TB of training data and running make to build the model on their own cluster of GPUs?

        • villish 16 minutes ago ago

          I want to caution against this line of thinking. NSA's fast16 program silently altered data during nuclear simulations, and that is also possible within open weight models. There is no reason to think there is anything like that currently, it also cannot be dismissed. Open source would include the training data so you could create a similarly capable model.

        • dymk 31 minutes ago ago

          The weights are the output artifact, the training corpus and system are the source.

          • culi 18 minutes ago ago

            Kimi publishes their code and a technical report on their methodology. But I think the weights are still important. It means anyone with the resources could run the same model on their own hardware.

          • charcircuit 10 minutes ago ago

            The training corpus + training code is just an automated editor for the weights. It would be like requiring the source code for Visual Studio for software made within it to be open source.

            Weights are not an output artifact no more than source code is. There is never a moment where you can claim that it's done. As requirements change new ways to change the weights / code come up. With different projects you might want to import the weights / code into a bigger model / codebase.

    • fnordpiglet an hour ago ago

      Interestingly the reality of the open source release is they are opening up to full distillation by the closed source model providers at a deeper and more fundamental level. If anything the open sourcing will help Anthropic and open ai ladder up faster. Open source has always been about mutual cooperation towards a goal and has never closed the door to commercial success. All the hand wringing about open weight models putting closed providers at a disadvantage doesn’t get what working in the open actually does for commercial interests - it is like science in the open - it enables and lifts all boats. Likewise commercial success doesn’t close the opportunity for competition or more open source work - it’s the economy of activity and competition that matters overall. When things stagnate is when closer concerns turtle up and collude on not competing for each others turf.

      The future is good and better for everyone the more work is in the open and the more work is in the commercial space. It’s good all around.

      • skeledrew 25 minutes ago ago

        The closed labs don't really benefit unless the open model has something extra they don't have though. Meanwhile the open model dilutes their customer base and seriously cheapens their offering (which is a heck of a good though IMO).

        • fnordpiglet 11 minutes ago ago

          Because the open model might have been distilled from outputs of the closed model doesn’t mean they are architecturally equivalent. There is almost certainly innovations in architecture present in the open models that the closed labs didn’t think of. It also is almost certainly true that they aren’t completely built out of a distilled corpus, that reinforcement is equivalent, etc. Therefore closed labs will also benefit from being able to inspect in totality the architecture, activations, weights, and be able to train against it at scale in an ensemble of other models and their internal work.

          The only way there is no benefit would be is if the open models are literal copies of the closed model, which unless there was direct theft, is highly improbable.

      • linkregister 39 minutes ago ago

        I'm happy that Kimi K3 is indeed SotA and its open weights are due to be released soon.

        It's also true that Moonshot and other labs distill from Claude. This has been reported on extensively. I don't think there's any alpha for Anthropic distilling from this model. I do not mean to discount the tremendous amount of innovation regarding MoE and quantization that Moonshot has accomplished. But its training with synthetic data is essentially distillation from frontier labs.

        • fnordpiglet 8 minutes ago ago

          When I say distill I also mean mine it architecturally for insights but I doubt seriously the model training is entirely distillation of Claude, it’s almost certainly a mixture of both original corpus and reinforcement as well as distillation. I think it’s a little condescending to imply that these new open models are cheap ripoffs with nothing original to them. These teams and labs are top tier as well, working under unreasonable constraints imposed by the USG. That’s a powerful combination for creativity.

    • mikae1 an hour ago ago

      For regular chat users it's $19 while Claude is $20...

      • AlexErrant an hour ago ago

        $20 users don't get access to Fable. It's $100+ tier only.

        • skeledrew 30 minutes ago ago

          I'm a $20 user and was given $100 extra usage credit today, "for Fable" (I'll be sticking to Sonnet and sometimes Opus TYVM).

        • mikae1 an hour ago ago

          True. However, I believe most non-programmers don't need access to the fanciest model, but just want to use a good LLM without constant nagging about usage limits. Then 19 vs. 20 is true?

          • culi 13 minutes ago ago

            No "19 vs 20" is not true for "Kimi vs Fable". It might be true for "Kimi vs Opus" or whatever. Anyways looking at pricing plans is not a good way of comparing the price of LLMs. It makes more sense to look at cost per token:

              Kimi K3: $3/$15 (input/output)
              Fable:  $10/$50
          • potwinkle an hour ago ago

            Still no. If you're only getting Opus-class you can still end up paying less by just using an equivalent Chinese model on OpenRouter.

            • mikae1 40 minutes ago ago

              Not talking about me or HN users in general. These companies likely need regular peeps to begin using their services in order to become profitable. Not sure these are the ones who will buy from OpenRouter.

    • OutOfHere an hour ago ago

      Or biology or chemistry.

    • estearum an hour ago ago

      hmm almost like the exact race-to-the-bottom + arms-race dynamic all the doomers have been warning about

      • andrewmutz an hour ago ago

        One man's offensive penetration tool is another mans defensive tool. In the recent HuggingFace/OpenAI incident the safety controls stood in the way of the defenders, not the attackers:

        https://www.thestack.technology/hugging-face-hacked-turned-t...

        • estearum an hour ago ago

          You're presenting further evidence of lack of effective control over these systems as... a mitigating factor...?

          • californical 44 minutes ago ago

            Why do you assume they're disagreeing with you?

            • estearum 41 minutes ago ago

              > One man's offensive penetration tool is another mans defensive tool.

              seems to suggest the author believes there's some intrinsic equilibrium

              Which,

              1) is definitely not proven and not guaranteed (open to proofs otherwise, not pithy sayings that have zero normative effect on reality)

              2) is apparently "supported by" further evidence of lack of effective control, which does not feel like equilibrium whatsoever

      • toomuchtodo an hour ago ago

        Race to the bottom for the investors, utility for the rest of us. The future is already here, it’s just not evenly distributed yet (Gibson).

      • skinfaxi an hour ago ago

        Doom for whom?

        • estearum an hour ago ago

          Well considering it was just a few months ago that OpenAI had the brilliant idea of connecting these systems to autonomously run an actual protein synthesis wetlab, very possibly all of us.

  • JSR_FDED 41 minutes ago ago

    Very interesting. They test Kimi K3 and Fable on a set of approx 1000 tasks grouped into 5 areas (SWE, Legal, etc).

    They put a router model in front that predicts whether Kimi or Fable is going to give a better cost for a correct result. (They believe that ultimately such a router model should be continuously trained on your own workloads so it makes the best decisions for you).

    Their router chose Kimi the majority of the time (72% in one category, all the way to 96% in another category), leading to cost savings in every category (from 1.5x to 50x depending).

    • crazylogger 16 minutes ago ago

      > Oracle routing is a method for measuring the best theoretical performance by running the task through each model and then picking the cheapest correct option (the cost/performance ceiling).

      Their "router" is an oracle reference point where they choose the lower cost model after running both and therefore knowing who passed the test. The cost savings part is only Fireworks theorizing what would happen if an equivalent predicting router exists. That's a big if.

    • eli 24 minutes ago ago

      There are a bunch of routers like this eg https://openrouter.ai/openrouter/auto

  • luciana1u 36 minutes ago ago

    we put a router model in front of two other models so the router can decide which model is better at deciding things. next we'll need a router for the router and eventually the entire internet is just routers routing routers to other routers

  • mattvr an hour ago ago

    Anyone have routing harnesses like this describes with Claude Code? Or other good routing platform recommendations?

    (yes, I know this article is about an oracle router)

  • nharada an hour ago ago

    Is there something specifically with Kimi that's better here? As far as I know Kimi pricing is about the same as Sonnet 5 -- what happens if you use that model and Fable instead? Or Grok 4.5 which is even cheaper?

    • DennisP 29 minutes ago ago

      The article says Kimi is better at some things than Fable. That's probably not true of Sonnet.

    • ralusek an hour ago ago

      One benefit of an open source one is that you can, as a large corporation, run it "locally" within your own data center. Even fine tune it.

  • Buttons840 32 minutes ago ago

    I have several thoughts about this, which I'll just iterate:

    1) US export bans have made it so that Chinese companies have to compete using less-than-state-of-the-art hardware. This has forced Chinese companies to build more cost efficient models. Whereas, US companies have moreso tried to be state of the art by spending more money than anyone else on state-of-the-art hardware.

    2) Xi Jinping has called for more open AI models (not to be confused with the closed models of OpenAI), and I'm happy to see a powerful world leader advocating for open-weight AI models. Whereas, the US seems likely to just ban models.

    3) My impression is that, if China surpasses the US in AI development, there will basically be nothing that the US does better than the rest of the world--except for military spending--we spend a lot, but we don't necessarily spend well (something something Iran). I mean, if the US is no longer a tech leader in the world, like... what are we a leader at? Manufacturing? Healthcare? LOL. Are we a leader in any industry or by any metric? I wonder if China is attempting to remove the last jewel in the USA's crown with these AI releases.

    4) It must be refreshing for companies to have access to a new model that isn't going to get pulled because the government bans it 2 days after release. And it's open-weight so it wont go away--amazing--what a shift in the market.

    5) If it becomes clear that open-weight models are the future of AI, will that pop a huge bubble in the US economy? Maybe. But, on the other hand, these companies aren't just training AIs, they are also building data centers which will remain valuable no matter what happens.

    • shdh 24 minutes ago ago

      Implying that AI is the last jewel of the USA is too simplistic and ignorant.

      People are paying for tokens, that will likely continue even if open weight wins.

  • johnhess 21 minutes ago ago

    Was this an out of sample test of the router or was it trained on these specific use cases/eval suites?

  • guessmyname an hour ago ago

    Why SoTA (uppercase “T”) instead of SotA (lowercase “T”) ?

    “State of [T]he Art” versus “State of [t]he Art”.

    If not SotA then at least SOTA, which is more accurate.

    • sim04ful an hour ago ago

      The pedantry of a HN comment is always a sight to see.

      • _carbyau_ 43 minutes ago ago

        Not wrong though. Technically correct and when it comes to communication, also seems like a reasonable query.

        In a world of so many acronyms, details matter.

    • lexandstuff an hour ago ago

      It should be SotA.

      • jamesinmn 40 minutes ago ago

        Does that make DeepSeek V4 Flash MiniSotA? This dev in the Twin Cities would like to know.

    • skeledrew 24 minutes ago ago

      I suspect typo.

    • pishpash 39 minutes ago ago

      An cross of SoT (source of truth) and SotA (state of the art). SotA looks weird, admittedly.

  • arjie an hour ago ago

    Interesting. So the latest in the technology now is this model routing thing. Cursor estimated Composer + Fable works much better than Fable alone. And here K3 + Fable is supposedly better. Interesting.

  • stingraycharles an hour ago ago

    As always, benchmarks rarely paint the whole picture. It also seems like this article is somewhat biased, eg when Fable and Kimi are close but Fable wins it’s “dead heat”, but when Kimi wins it’s “Kimi wins”. GPT 5.6 seems to be missing as well.

    I am really eager to give Kimi K3 a try, but I’ll reserve my judgement until I’ve worked with it for at least a few days.

    • rogerrogerr an hour ago ago

      The apparent bias may be explainable as it’s not remarkable for OpenAI or Anthropic to be slightly ahead. It _is_ remarkable for an open weights model to be better than the closed models from the trillion dollar (allegedly) companies.

  • felipeerias an hour ago ago

    Mythos/Fable was the state of the art back in March, if not earlier.

  • sbinnee 40 minutes ago ago

    Openrouter also features routing. Routing is indeed an option if you don’t need consistent behavior and allow switching models.

    • JSR_FDED 26 minutes ago ago

      It would only be inconsistent if the router chose different models for the same task.

      Considering that of 89 terminal tasks there were 11 that only Kimi got right, and 7 only Fable - having a router gives significantly more consistent results if you define consistent to mean correct.

  • jrflo an hour ago ago

    Hmmm, a company that hosts open models is telling us how good open models are...

    • JSR_FDED 33 minutes ago ago

      They share their methodology and results. I learned things about the relative strengths and weaknesses of Kimi and Fable I hadn’t seen anywhere else. Should being in the model hosting business disqualify them from sharing?

      • jrflo 9 minutes ago ago

        Doesn't disqualify them, but it may call into question their results seeing as they have a potential conflict of interest.

      • Avicebron 25 minutes ago ago

        People are allowed to say, "hey we make money off of this thing, it's cheaper and almost as good as the thing we can't money off".

        Other people are allowed to call them on that.

    • mrinterweb an hour ago ago

      The don't only host open weight models. Also, why not promote this. If Fireworks thinks this big news might convert some new business doesn't make it not true.

      • jrflo 8 minutes ago ago

        I'd be doing the same thing if I were them as a marketing move

      • 30minAdayHN an hour ago ago

        and in fact, it would be detrimental to business, if they wrongly promote Kimi, as they would pretty soon lose trust with their users

    • yogthos an hour ago ago

      Anybody who's tried it knows that what they're saying is true though.

  • hawtads an hour ago ago

    For model routers, do they have to retrain the routing model every time a new LLM is released?

    • pishpash 36 minutes ago ago

      Maybe a continuous evaluation process, why is there anything to train?

  • Sphax an hour ago ago

    what are my options if i want to use a router like this ? who provides one ?

  • OutOfHere an hour ago ago

    They forgot to compare and incorporate GPT-5.6-Sol.

  • lvl155 an hour ago ago

    It is not SOTA. Give me a break. Sure, run it on Cerebras to get speed but that’s pretty much its advantage.

    • nozzlegear 17 minutes ago ago

      Why do you think it's not SOTA?

  • refulgentis an hour ago ago

    AI written drivel by an inference provider, hyping an open model that isn’t open, otherwise the inference provider would be hosting it, and the inference provider also does not have this router they trumpet available, in any form.

    I’d be amenable to it but I’m already aware of them lying about token throughout by 50-100% on AA and on Twitter, even when you get a dedicated server.

    Just too much lying piled up to extend courtesy here, even though you’re my favorite open model provider. Please, get your house in order, and if the house is so big that marketing drove this set, ask them to slow down a bit and market things that people can actually use / you actually provide.

    • villish 3 minutes ago ago

      I'm not sure why you're so dismissive about this model. It isn't currently open, but it will be. It was slow when I used it so i'll give you that. There will be other providers with better performance.

      My only gripe was that it takes a very long time to think. It's a good model otherwise.

    • JSR_FDED 21 minutes ago ago

      > hyping an open model that isn’t open

      Moonshot has committed to release it end of the month.