2 comments

  • docshock 34 minutes ago ago

    Yes. The sweet spot for me was determined by what I already had and what was available at or near MSRP. I managed to find a 5080 last month at Walmart. Because I already had 96 GB of system RAM I settled on a mixture-of-experts model for my use case. I opted for NVIDIA Nemotron 3.5 Bolt 74B.

    Yes, I continue to use cloud and frontier models. The cloud model is served as a backing model through NVIDIA Switchyard.

    At this point I am using Claude pretty much for research only.

  • Aurornis 5 hours ago ago

    Primary question is: What are your goals and what will you use it for?

    I have a 128GB platform that I use for local LLMs. It's fun, but even when I'm pushing the limits of what it can hold, I have to be honest that it's not a replacement for a $20 Claude account. To put it in perspective, the best results I've been able to get out of my local system are still less than the mid-tier models from OpenAI and Anthropic (Terra or Sonnet).

    If privacy is your goal then that doesn't matter. A local LLM setup delivers that. If saving money is your goal, enjoy one of the frontier model subscription plans and wait for hardware prices to come down again.

    Before you buy anything, you should determine the best model that would run on that hardware. Then spend $20 on OpenRouter using that model for the hardest task you can think you would use it for. Then remember that the version of that model you run locally will be quantized, so quality will be even worse than that.

    For some people do some work, Qwen3.8-27B is all they need, which opens up a lot of options. For complex work, the gap between the open models that fit on consumer hardware and even the mid-tier models from a $20/month plan is large. You should aim to understand this gap and how it relates to your work before you spend $3000 or more on hardware for it.