AI means it has npu, Max+ is marking the memory channel, PRO is a normal label for chips that have extra security baked in, it has been this way since forever.
To expand, positional definition of adjectives tends to be a bad marketing scheme.
Nobody knows that pro and max mean specific things.
If AMD needs to create a full cross-product of their hardware feature set, because they intend to have a SKU for every possible combination, they probably need a better scheme.
Say what you want about Celeron/Pentium/i3/i5/i7, but it mostly got the idea across (except on the occasions Intel hijacked it to upbrand SKUs).
To be fair - I'd much rather the name be overwrought but clear (once you know or have the decryption key to translate) than to have to look up an even uglier chart that i have ~no chance of memorizing to go 'ok, the Tiberian has X, Y, Z and Q and the Cyrian has Q, Y, but Z is nerfed'
I have a framework desktop w/ 128GB that I bought last Christmas and if I’m looking at it right it costs $2000 (CAD) more now because of the RAM shortage (and in any event is apparently out of stock). Would love to have 192 GB but I’m not sure I can justify buying another.
Does anyone have a sense of how this might progress, e.g if I can get a 256 or 512 GB in a year if I wait. In any case I’m jealous this exists and I don’t have one.
One last thing, I assume this isn’t exclusive and there will be other builds with this same config same as current Strix Halo?
They are estimating a 25% shortfall in supply remaining in 2030, even with Chinese companies ramping up DDR5 supply.
It's not looking great - the RAM producers need more of the same machines that other semi-conductor manufactures need and the suppliers of those seem unable to increase production.
> the RAM producers need more of the same machines that other semi-conductor manufactures need and the suppliers of those seem unable to increase production
I suspect that it is very, very profitable for them not to increase production and instead increase prices.
This is one of those cases where the free market fails and governments should step-in. IP laws are there to elevate the arts and sciences, not stifle them in the name of profit.
I mean, that's what the US constitution says very plainly. It does seem to be a pretty unpopular take these days though.
"To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries."
Yes, others should be able to do this too, and the change is larger memory chips.
The next big step is Medusa Halo, which will have a 384-bit LPDDR6 interface. Those should be able to support 256GB at release, with 512GB coming later with larger chips (But I don't know to what extent people should trust the memory vendor roadmaps.) I'm not sure if they will be out in a year. Probably will be in a year and half.
Unfortunately we don't have any indication of memory pricing coming down in the next year, and many indications pointing to memory pricing continuing to increase pretty substantially over the next six months, especially for the LPDDR5X we use in Framework Desktop.
The market research I have seen indicated DRAM prices will not come back to “normal” until 2030 at least and probably 2031. You can wait, but it will probably be longer than a year
Are you generally happy with it? What do you mainly use it for?
I'm kinda tempted by the frameworks because they are somewhat energy efficient and compact (which I both value highly!) but my concern is that the GPU is on the low end for 1440p gaming (while being barely price-competitive with a self-built Ryzen9950 + 9070XT combination).
I use it exclusively as a server for LLM inference - first more for research but have been using it for coding now that there are sufficiently capable models that run on. For that I’m very happy.
Most games play just fine at 1440 or 4K with upscaling.
Cyberpunk ray tracing is a bad experience, but I just finished 007 First Light on ultra settings with an 5k monitor using FSR and it was 45+ fps the whole time. Targeting 4K with fsr was 60fps.
I’m not a hardcore gamer or a pro reviewer, but it mostly just works without having to change much. If I were a hardcore gamer I think I would go with a better setup.
My big complaint for gaming is that they never enabled HDMI CEC. Apparently the pins are wired but they never shipped the firmware.
Likewise here, my setup costs 2k+ more eur as well. Given that memory bandwidth remains the same I don't think it's worth buying if you have the 1st gen framework desktop. Usecase caveats apply.
what I read in I think the direct AMD press, it's essentially like a 10% upgrade to compute but mostly ram.
I agree, this is the direction AMD was going in to larger attached memory and they got hit by the memory cartel pricing; They likely were going to hit 256GB instead of this weird glitch in the sizing.
So, yes, of course they're going to hit higher memory sizes; but since they models are meant for laptops and to get the speed you want for inference, they're soldered, you have no real options.
Framework in particular might have a high value on ebay as I assume this will be a drop in replacement for their existing motherboard.
I typed my (masked) email into the input, pressed tab and enter, it redirected me to some cloudflare captcha marketing site. I thought at first it didn't want the masked email, but then it worked when I clicked with my mouse. Apparently there are 3 invisible focus targets in between!
Dear people who create websites, these things are important, they should work!
I think it works only on type text? I tab and enter out of habit, but to be fair, enter without tab works here. But then again (and many will feel I'm being nit-picky but you're just hearing from someone who had to conquer a lot of a11y audits at job, so just let me have different priorities), the whole node gets replaced without any announcement, Voiceover being left clueless about what happened. NVDA would probably be the same, but I'm on vacation and cannot test on windows right now.
This is a simple form, not a complicated thing, and it's the website of a company I love, so I'm paying extra attention perhaps.
Im less interested in the memory than the memory bandwidth. The current system with 128GB can load pretty big models, but its meaningless unless you want to wait 40 minutes per prompt.
I've had best results with Qwen3.6-35B-A3B, which uses 40GB of memory, but only uses 3 billion parameters per token which helps with throughput.
Until memory bandwidth significantly improves I just can't see myself wanting to use all that memory. Unless it's just to keep a wide variety of models in memory.
I run Qwen3-Coder-Next-UD-Q4_K_XL and other than the initial wait to initialise context (which takes less than 2 minutes) subsequent prompts return in less than a minute, usually less than 30s.
If your performance is significantly slower then you are probably doing it in CPU - there was some fiddling required to get it to use GPU (I use llama.cpp)
For those wondering, the AI Max 395 has around 256GB/s of memory bandwidth, whereas this new 495 has 273GB/s. So a very modest improvement in bandwidth.
I use exactly the setup you describe; it can handle 1-3 agents working if the agent work has IO delays. with MTP models, prefill is quite fast.
Not sure what damage you have, but no one waits 40 minutes per prompt or even a minute; The A3B model loads within 10 seconds and a simple response in opencode is maybe at most a minute, then catches up in 3-5second bursts depending on IO.
I put dynamic context pruning into opencode and tweaked it for 45k-85k context before it shrinks; this lets me get into 500k token sizes and fixed on medium sized github repos.
There's a chance this comment is a skill error:
1. USe llamacpp with a MTP model
2. Use reasoning-budget and reasoning-message
3. Tailor your agent to use the reasoning-message to use subagents and dyanmic compaction.
---
Now you have a reasonable coding agent for cloning, building and extending any github project I've seen so far.
It has all the backends you want. Just a script rebuilds it. Then I just put it in a docker compose and whenever we want to update it, we pull the repo and rebuild it.
I just a models.ini like:
version = 1
```
[*]
n-gpu-layers = 999
threads = 28
flash-attn = on
mlock = off
mmap = off
fit = off
warmup = on
jinja = true
direct-io = on
cache-prompt = true
cache-reuse = 256
models-max = 1
models-autoload = 1
timeout = 600
ctx-checkpoints = 256
cache-ram = 0
mlock = on
kv-unified = on
no-context-shift = on
swa-full = on
[Q36-35B-A3B]
model = Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf
mmproj = mmproj-BF16.gguf
rope-scaling = yarn
rope-scale = 1.14441
yarn-orig-ctx = 262144
override-kv = qwen35.context_length=int:1000000
ctx-size = 300000
reasoning = auto
reasoning-budget = 421
reasoning-budget-message = you're reasoning too much; compress context and use subagents to investigate further
spec-type = draft-mtp
spec-draft-n-max = 3
batch-size = 16001
ubatch-size = 16001
cache-idle-slots = on
temp = 0.6
top-p = 0.95
min-p = 0.00
top-k = 20
repeat-penalty = 1.1
presence-penalty = 0.0
```
Using subagents, you keep the context size from bloating, and add tooling for the context compaction and it can keep looping. You probably want to clean up opencode's agent; back in the "Prompt Engineering" days there's a lot of unnecessary fluff. The goal of a prompt should be to get the model in the correct starting place, and not, try to give it everything it ever needs to do what you want it to do.
I have an NVIDIA Spark thingy (the ASUS one) and it's the same problem there, though the prefill side is superior to the Ryzen ones.
But I actually think 128GB is too little. There are some compelling models that are above what can fit in that at reasonable quants (e.g. DeepSeek V4 Flash) but could if the system was 256GB.
If RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common. As it is I think it will be 10 years before >128GB becomes feasible on a regular consumer machine for normal people again.
You can run DeepSeek Flash just fine on 128GB using lower quants. Antirez' DwarfStar supports it really well. The bigger the model, it seems, the less it is affected by high quantization. I know some people are working on quantizing GLM 5.2 so it can fit into 128GB (using additional tricks to select only some experts, etc). There's a lot going on.
> RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common
I agree but unfortunately that speaks to the depth of demand right now. Even as more capacity comes online (if it does) I suspect it will all get used up without prices needing to drop. Hopefully on the other side of this 10 years from now it means RAM will be insanely abundant and we’ll all have machines with a couple TB as the same inflation adjusted price as say 16GB a few years ago. Considering single digit MB of RAM was the norm in the early-mid ‘90s its not so far fetched.
I understand their reasoning but it’s still a shame Framework opted for a proprietary motherboard instead of mini-ITX with a socket CPU and GPU. I value repairability over a little bit of extra performance.
This is correct, we used a standard Mini-ITX form factor, and we've seen a number of customers pick up just the Mainboard and drop it into standard ATX cases. The power supply is also a standard FlexATX PSU.
IIRC, Framework did ask AMD to explore a more repairable approach with socketable CPU and RAM for these chips and they did but came back and said it wouldn't perform well enough that way.
It's not really about performance, it's about signal integrity. AMD said that even with LPCAMM2, you just wouldn't be able to send a clean enough signal from all these different memory channels to the chip, and they insisted on a soldered design because they just couldn't make anything else work.
Technically it's not wrong to say it's about performance in the sense that if you clocked the RAM slow enough you probably could maintain signal integrity, but we're not talking some small hit to performance here, it'd probably be more like a multi-generational drop in memory bandwidth.
And that is where Framework should have stepped in and said “we as a company value repairability, so we will take the performance hit to make the product easier to repair”
It doesn't work that way. AMD said they simply couldn't do it. It wasn't about taking some modest performance hit, it was that they couldn't make the chip function with modular memory due to signal integrity problems.
Why not put a big fat Ryzen 9 chip in it? Most people use their computer for browsing the web, playing some games, doing some spreadsheets. A standard CPU would have been more than powerful enough for that and the Desktop would be more repairable.
The whole point is that these APUs present a pretty unique value proposition (lowish performance GPUs with massive amounts of RAM attached), and framework is afaik the only vendor shipping them on standard mini-ITX boards
And the whole point of Framework is to have great repairability on their products, better than the competition even. Shipping products with soldered RAM and CPU goes against that ethos.
There is no socketable Unified Memory platform. The entire point of this SoC is that it has Unified Memory. The Framework Desktop has a Mini-ITX board, but you MUST use a solderable SoC with HBM stacked memory to get a Unified Memory outcome. For local AI workloads, that's the only part that matters, which is the entire point of wanting 192GB of RAM available on an APU.
I like the repairability and modularity of my Framework 13 laptop, and I still bought a maxed out MBP M5 Max because for local LLM, unified memory is all that matters.
If this CPU is not socketable by design, why not pick a CPU that is? There are plenty of options. Repairability is more important than raw speed. That is why people pick Framework. It’s not the best machine at everything but it’s easy to repair.
You fundamentally cannot do things on the system you are envisioning that you can do on a Framework Desktop, or DGX Spark, or MBP M5 Max. Simple as that. Repairable modular laptops were a fairly rare thing when Framework came out. A repairable modular desktop is just a normal desktop computer. A desktop computer in a compact form factor that can run AI workloads though is a much rarer thing.
My computer with socketed CPU had no issues being a computer. It’s so strange for Framework to make a machine you can’t repair. It’s the Framework Desktop. Why not create the best possible repairable Desktop instead of using some niche CPU soldered to the board? Not everyone has AI workloads. Some people just browse the web a little.
The people who buy a Framework Desktop with 192GB of RAM (or in my case an M5 Max MBP with 128GB of RAM). I don't know what argument you're trying to make here? You are saying this product is pointless, the point of this product is clearly stated and obvious, and now you're acting like its not relevant.
If you don't want to engage in a reasonable discussion, you can get bent.
Framework is going for a computer OEM market where they need to make margin, hence the soldered-on components and relatively limited options. People in the pre-built computer market overwhelmingly don't care about replaceable components, because anyone else will just buy components and build a PC, for both the greater market of options AND overall lower price than buying pre-built (if they don't overspec).
You literally can't make this machine without soldered components. You don't understand this computer's design goals. It's been 1.5 years since they announced the Framework Desktop. I don't get how people are still confused about this.
Mmm.. AI395 at 3.5k ( and out of stock mind ). And I still want to give it a shot when it is out. I suppose I just explained to myself the ridiculous jump in prices. The demand is crazy.
Woof. What's that going to cost? And will we actually be able to buy one, because the 128GB model has been sold out pretty much since the LLM craze started...
We can't share pricing yet, but certainly will before we open orders. The 128GB has been selling well, and we just re-opened orders with stock landing within two weeks.
Rumored M5 ultra bandwidth is apparently 1100 GB/S (M3 Ultra is something like 880). This is 276 GB/S so they’re not really in the same league unfortunately. This will be considerably cheaper though.
This machine needs better networking options to be useful, such as built-in 100Gbit QSFP28. There is not enough space between the x4 PCIe slot and the power supply for a NIC to fit, and even if there was, you'd have to cut a hole in the back of the case to get at it.
Tl;dr: think hard about what you’d use this much RAM for in a desktop setting and do some research about how people like the 395 for your use case. Not all use cases work well.
Anyone who thinks they are going to serve some 100+ GB LLM locally, remember that memory bandwidth becomes a key limitation for large models. While you might be able to load a model, token generation can be very slow. MoE models like Qwen3.5-122B-A10B work decently fast, but dense models of a decent size are slow and you won’t want to use them.
I’ve got a 395 system, and found that I’m quite happy with Qwen3.6-35B-A3B, generating at around 50 t/s, but the dense 27B model is 20-25 t/s and that’s the lower limit I’m willing to tolerate. So a 70b dense model is just not going to happen. That means you can’t really use that much RAM.
A reasonable use case is to have multiple smaller models loaded- you can have an image generation model loaded along with the text model. Or you can use this computer for development simultaneously with serving LLMs. Those ideas work okay. But trying to load up a single giant model is going to test your patience.
> A reasonable use case is to have multiple smaller models loaded- you can have an image generation model loaded along with the text model.
Lemonade server updated yesterday for an Arena model / ai routing.
---
The Lemonade local AI server allows easily deploying LLM workloads across Ryzen AI NPUs, Radeon GPUs, and CPUs with ease under both Windows and Linux. The most significant change with Lemonade 11.5 is the completion of the Lemonade Router that can be used for automatically routing queries to relevant models based on defined policies -- or even LLM-as-a-router for using a small LLM to in turn determine which model to route a particular request.
The Lemonade Router allows steering requests based on rule, classifier, semantic similarity, or LLM-as-router policies. The policies are defined in JSON files or can also be authored via the Lemonade GUI. More details on these router capabilities can be found via this commit.
Lemonade 11.5 also introduces a server-side job engine to let clients post multi-step recipes and managing them via /jobs endpoints. From these endpoints the muilti-step recipes can be paused / interrupted / resumed / deleted. And another one is that the lemond daemon can now act as an MCP client host to connect external stdio MCP servers.
Memory bandwidth certainly is the bottleneck on my 395+ 128GB. It's been a great server. Think I'll wait another year maybe hopefully the prices come down and there's one more gen of hardware and local llm gets even better. Excited for this!
Somewhat on-topic, if anyone here's received their Framework 13 Pro yet (which I believe began shipping this month), I'd be happy to hear a first-hand review.
the thing about ECC memory is that it doesn't support reporting of ECC errors, so it ends up just hiding the fact that your RAM has started to go bad. this is good if you replace RAM every n years, but not if you intend to use the RAM until it starts degrading.
>the thing about ECC memory is that it doesn't support reporting of ECC errors
Wait that. It does, and that's the point. Although the CPU and motherboard do of course also have to support it, but with AMD it is possible. I do it on a 9800x3d.
One bit errors are always both detected and corrected. Multibit errors are mostly at least detected.
Ryzen AI Max Plus Pro 495
How does a CEO read this and not immediately fire their entire marketing team?
I know right. Where’s the «Elite»?
Don't forget the X's as well, i'm looking at you Amd RX 7900XTX
What is this, Dwarf Fortress?
Because it was the CEOs idea?
They were just leaving room for the "Ultimate" model, to be released next year.
Dell did this too with their workstation laptops, Dell Pro Max 16 Plus. :/
We're running out of space on our e-commerce UI elements and physical labels to fit some of these names.
Schemes like this just indicate they've joined the USB IF.
Why would they? It's a very scalable naming scheme, there's even room for Max Plus Pro Ultra, with 256GB of RAM.
More numbers and letters and phrases = more excite! much customer!
Agreed, what the heck were they thinking, it's obvious it should've been Ryzen AI Max Plus Pro Ultra 495.
It’s like after 10 years of DLCs where every items has 5 suffixes and prefixes
AI means it has npu, Max+ is marking the memory channel, PRO is a normal label for chips that have extra security baked in, it has been this way since forever.
I don't see anything wrong with the name.
I choose to believe this comment is satire.
You literally answered what is wrong with the name.
They did not, can you explain what do you mean?
For those of us with feebler minds than you or him, the name is just full of qualifiers that make no sense.
To expand, positional definition of adjectives tends to be a bad marketing scheme.
Nobody knows that pro and max mean specific things.
If AMD needs to create a full cross-product of their hardware feature set, because they intend to have a SKU for every possible combination, they probably need a better scheme.
Say what you want about Celeron/Pentium/i3/i5/i7, but it mostly got the idea across (except on the occasions Intel hijacked it to upbrand SKUs).
So why do so many of major successful product corporations use this kind of naming scheme?
It is possible to be successful in spite of one’s decisions
To be fair - I'd much rather the name be overwrought but clear (once you know or have the decryption key to translate) than to have to look up an even uglier chart that i have ~no chance of memorizing to go 'ok, the Tiberian has X, Y, Z and Q and the Cyrian has Q, Y, but Z is nerfed'
I have a framework desktop w/ 128GB that I bought last Christmas and if I’m looking at it right it costs $2000 (CAD) more now because of the RAM shortage (and in any event is apparently out of stock). Would love to have 192 GB but I’m not sure I can justify buying another.
Does anyone have a sense of how this might progress, e.g if I can get a 256 or 512 GB in a year if I wait. In any case I’m jealous this exists and I don’t have one.
One last thing, I assume this isn’t exclusive and there will be other builds with this same config same as current Strix Halo?
On RAM supplies, the best estimates I've seen are https://www.tomshardware.com/pc-components/dram/cxmt-close-t...
They are estimating a 25% shortfall in supply remaining in 2030, even with Chinese companies ramping up DDR5 supply.
It's not looking great - the RAM producers need more of the same machines that other semi-conductor manufactures need and the suppliers of those seem unable to increase production.
> the RAM producers need more of the same machines that other semi-conductor manufactures need and the suppliers of those seem unable to increase production
I suspect that it is very, very profitable for them not to increase production and instead increase prices.
This is one of those cases where the free market fails and governments should step-in. IP laws are there to elevate the arts and sciences, not stifle them in the name of profit.
> IP laws are there to elevate the arts and sciences, not stifle them in the name of profit.
Hmmmmmmmmm
I mean, that's what the US constitution says very plainly. It does seem to be a pretty unpopular take these days though.
"To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries."
Yes, others should be able to do this too, and the change is larger memory chips.
The next big step is Medusa Halo, which will have a 384-bit LPDDR6 interface. Those should be able to support 256GB at release, with 512GB coming later with larger chips (But I don't know to what extent people should trust the memory vendor roadmaps.) I'm not sure if they will be out in a year. Probably will be in a year and half.
Unfortunately we don't have any indication of memory pricing coming down in the next year, and many indications pointing to memory pricing continuing to increase pretty substantially over the next six months, especially for the LPDDR5X we use in Framework Desktop.
The market research I have seen indicated DRAM prices will not come back to “normal” until 2030 at least and probably 2031. You can wait, but it will probably be longer than a year
Are you generally happy with it? What do you mainly use it for?
I'm kinda tempted by the frameworks because they are somewhat energy efficient and compact (which I both value highly!) but my concern is that the GPU is on the low end for 1440p gaming (while being barely price-competitive with a self-built Ryzen9950 + 9070XT combination).
I use it exclusively as a server for LLM inference - first more for research but have been using it for coding now that there are sufficiently capable models that run on. For that I’m very happy.
Most games play just fine at 1440 or 4K with upscaling.
Cyberpunk ray tracing is a bad experience, but I just finished 007 First Light on ultra settings with an 5k monitor using FSR and it was 45+ fps the whole time. Targeting 4K with fsr was 60fps.
I’m not a hardcore gamer or a pro reviewer, but it mostly just works without having to change much. If I were a hardcore gamer I think I would go with a better setup.
My big complaint for gaming is that they never enabled HDMI CEC. Apparently the pins are wired but they never shipped the firmware.
Likewise here, my setup costs 2k+ more eur as well. Given that memory bandwidth remains the same I don't think it's worth buying if you have the 1st gen framework desktop. Usecase caveats apply.
what I read in I think the direct AMD press, it's essentially like a 10% upgrade to compute but mostly ram.
I agree, this is the direction AMD was going in to larger attached memory and they got hit by the memory cartel pricing; They likely were going to hit 256GB instead of this weird glitch in the sizing.
So, yes, of course they're going to hit higher memory sizes; but since they models are meant for laptops and to get the speed you want for inference, they're soldered, you have no real options.
Framework in particular might have a high value on ebay as I assume this will be a drop in replacement for their existing motherboard.
I typed my (masked) email into the input, pressed tab and enter, it redirected me to some cloudflare captcha marketing site. I thought at first it didn't want the masked email, but then it worked when I clicked with my mouse. Apparently there are 3 invisible focus targets in between!
Dear people who create websites, these things are important, they should work!
I shared this internally, and the team confirmed this is indeed a bug in our email form rendering. We've added it to the development backlog.
Thanks for caring, this is really cool hearing from you!
You shouldn't need to tab first, right? Enter on an input field should submit the form unless they go out of their way to break it?
I think it works only on type text? I tab and enter out of habit, but to be fair, enter without tab works here. But then again (and many will feel I'm being nit-picky but you're just hearing from someone who had to conquer a lot of a11y audits at job, so just let me have different priorities), the whole node gets replaced without any announcement, Voiceover being left clueless about what happened. NVDA would probably be the same, but I'm on vacation and cannot test on windows right now.
This is a simple form, not a complicated thing, and it's the website of a company I love, so I'm paying extra attention perhaps.
Im less interested in the memory than the memory bandwidth. The current system with 128GB can load pretty big models, but its meaningless unless you want to wait 40 minutes per prompt.
I've had best results with Qwen3.6-35B-A3B, which uses 40GB of memory, but only uses 3 billion parameters per token which helps with throughput.
Until memory bandwidth significantly improves I just can't see myself wanting to use all that memory. Unless it's just to keep a wide variety of models in memory.
I run Qwen3-Coder-Next-UD-Q4_K_XL and other than the initial wait to initialise context (which takes less than 2 minutes) subsequent prompts return in less than a minute, usually less than 30s.
If your performance is significantly slower then you are probably doing it in CPU - there was some fiddling required to get it to use GPU (I use llama.cpp)
Try poolside that came out yesterday https://news.ycombinator.com/item?id=49004937
or Qwen 3.5 122B A10B, both use more memory and still have experts sized for decent speed at the 395’s memory bandwidth at 4bit quantization
If you are waiting 40 minutes for a prompt on a Ryzen max+ you have no idea what you are doing.
For those wondering, the AI Max 395 has around 256GB/s of memory bandwidth, whereas this new 495 has 273GB/s. So a very modest improvement in bandwidth.
I use exactly the setup you describe; it can handle 1-3 agents working if the agent work has IO delays. with MTP models, prefill is quite fast.
Not sure what damage you have, but no one waits 40 minutes per prompt or even a minute; The A3B model loads within 10 seconds and a simple response in opencode is maybe at most a minute, then catches up in 3-5second bursts depending on IO.
I put dynamic context pruning into opencode and tweaked it for 45k-85k context before it shrinks; this lets me get into 500k token sizes and fixed on medium sized github repos.
There's a chance this comment is a skill error:
1. USe llamacpp with a MTP model
2. Use reasoning-budget and reasoning-message
3. Tailor your agent to use the reasoning-message to use subagents and dyanmic compaction.
---
Now you have a reasonable coding agent for cloning, building and extending any github project I've seen so far.
Are you on Strix Halo? If so, which llama.cpp backend are you using? ROCm on Strix Halo still seems incompatible with MTP models.
I'm able to use ROCm with MTP just fine (Qwen and Gemma).
I use Lemonade and have only the ROCm installed through it (ETA: their llama.cpp version is a fork, not sure what is different vs the main trunk).
yes. I build a docker file from this: https://github.com/kyuz0/amd-strix-halo-toolboxes
It has all the backends you want. Just a script rebuilds it. Then I just put it in a docker compose and whenever we want to update it, we pull the repo and rebuild it.
I just a models.ini like: version = 1 ``` [*] n-gpu-layers = 999 threads = 28 flash-attn = on mlock = off mmap = off fit = off warmup = on jinja = true direct-io = on cache-prompt = true cache-reuse = 256 models-max = 1 models-autoload = 1 timeout = 600 ctx-checkpoints = 256 cache-ram = 0 mlock = on kv-unified = on no-context-shift = on swa-full = on
[Q36-35B-A3B] model = Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf mmproj = mmproj-BF16.gguf rope-scaling = yarn rope-scale = 1.14441 yarn-orig-ctx = 262144 override-kv = qwen35.context_length=int:1000000 ctx-size = 300000 reasoning = auto reasoning-budget = 421 reasoning-budget-message = you're reasoning too much; compress context and use subagents to investigate further spec-type = draft-mtp spec-draft-n-max = 3 batch-size = 16001 ubatch-size = 16001 cache-idle-slots = on temp = 0.6 top-p = 0.95 min-p = 0.00 top-k = 20 repeat-penalty = 1.1 presence-penalty = 0.0 ```
Using subagents, you keep the context size from bloating, and add tooling for the context compaction and it can keep looping. You probably want to clean up opencode's agent; back in the "Prompt Engineering" days there's a lot of unnecessary fluff. The goal of a prompt should be to get the model in the correct starting place, and not, try to give it everything it ever needs to do what you want it to do.
I have an NVIDIA Spark thingy (the ASUS one) and it's the same problem there, though the prefill side is superior to the Ryzen ones.
But I actually think 128GB is too little. There are some compelling models that are above what can fit in that at reasonable quants (e.g. DeepSeek V4 Flash) but could if the system was 256GB.
If RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common. As it is I think it will be 10 years before >128GB becomes feasible on a regular consumer machine for normal people again.
You can run DeepSeek Flash just fine on 128GB using lower quants. Antirez' DwarfStar supports it really well. The bigger the model, it seems, the less it is affected by high quantization. I know some people are working on quantizing GLM 5.2 so it can fit into 128GB (using additional tricks to select only some experts, etc). There's a lot going on.
> RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common
I agree but unfortunately that speaks to the depth of demand right now. Even as more capacity comes online (if it does) I suspect it will all get used up without prices needing to drop. Hopefully on the other side of this 10 years from now it means RAM will be insanely abundant and we’ll all have machines with a couple TB as the same inflation adjusted price as say 16GB a few years ago. Considering single digit MB of RAM was the norm in the early-mid ‘90s its not so far fetched.
I understand their reasoning but it’s still a shame Framework opted for a proprietary motherboard instead of mini-ITX with a socket CPU and GPU. I value repairability over a little bit of extra performance.
Based on the product spec sheet[1] it appears that the Framework Desktop motherboard is mini-ITX, and that the chassis is[2] as well.
It’s fully possible that I’ve missed something, but it doesn’t appear to be proprietary to me.
[1]: https://frame.work/desktop?tab=specs
[2]: https://frame.work/products/desktop-case
This is correct, we used a standard Mini-ITX form factor, and we've seen a number of customers pick up just the Mainboard and drop it into standard ATX cases. The power supply is also a standard FlexATX PSU.
I’m unable to pass the human test on those links but if it’s true mini-ITX including standardized USB and IO headers that’s pretty cool!
IIRC, Framework did ask AMD to explore a more repairable approach with socketable CPU and RAM for these chips and they did but came back and said it wouldn't perform well enough that way.
It's not really about performance, it's about signal integrity. AMD said that even with LPCAMM2, you just wouldn't be able to send a clean enough signal from all these different memory channels to the chip, and they insisted on a soldered design because they just couldn't make anything else work.
Technically it's not wrong to say it's about performance in the sense that if you clocked the RAM slow enough you probably could maintain signal integrity, but we're not talking some small hit to performance here, it'd probably be more like a multi-generational drop in memory bandwidth.
And that is where Framework should have stepped in and said “we as a company value repairability, so we will take the performance hit to make the product easier to repair”
It doesn't work that way. AMD said they simply couldn't do it. It wasn't about taking some modest performance hit, it was that they couldn't make the chip function with modular memory due to signal integrity problems.
Then why not pick a CPU that is socketable? There are dozens of them on the market that Framework could have picked for their Desktop product.
None of them have something even close to the memory bandwidth or integrated GPU performance of the Ryzen AI Max processors.
They want a Apple M_ Max like machine.
Why not put a big fat Ryzen 9 chip in it? Most people use their computer for browsing the web, playing some games, doing some spreadsheets. A standard CPU would have been more than powerful enough for that and the Desktop would be more repairable.
There's a million machines that are just as upgradeable as anything Framework could make if you just want a big fat Ryzen 9.
What’s stopping you from building that yourself instead of having to go through Framework?
What do you mean?
The whole point is that these APUs present a pretty unique value proposition (lowish performance GPUs with massive amounts of RAM attached), and framework is afaik the only vendor shipping them on standard mini-ITX boards
And the whole point of Framework is to have great repairability on their products, better than the competition even. Shipping products with soldered RAM and CPU goes against that ethos.
There is no socketable Unified Memory platform. The entire point of this SoC is that it has Unified Memory. The Framework Desktop has a Mini-ITX board, but you MUST use a solderable SoC with HBM stacked memory to get a Unified Memory outcome. For local AI workloads, that's the only part that matters, which is the entire point of wanting 192GB of RAM available on an APU.
I like the repairability and modularity of my Framework 13 laptop, and I still bought a maxed out MBP M5 Max because for local LLM, unified memory is all that matters.
If this CPU is not socketable by design, why not pick a CPU that is? There are plenty of options. Repairability is more important than raw speed. That is why people pick Framework. It’s not the best machine at everything but it’s easy to repair.
You fundamentally cannot do things on the system you are envisioning that you can do on a Framework Desktop, or DGX Spark, or MBP M5 Max. Simple as that. Repairable modular laptops were a fairly rare thing when Framework came out. A repairable modular desktop is just a normal desktop computer. A desktop computer in a compact form factor that can run AI workloads though is a much rarer thing.
My computer with socketed CPU had no issues being a computer. It’s so strange for Framework to make a machine you can’t repair. It’s the Framework Desktop. Why not create the best possible repairable Desktop instead of using some niche CPU soldered to the board? Not everyone has AI workloads. Some people just browse the web a little.
Can it run a local LLM with 35B+ parameters using GPU acceleration for reasonable token performance? If not, then you're being obtuse.
How many people use a desktop for a local LLM with 35B+ parameters? Maybe 0.0001% of computer users?
The people who buy a Framework Desktop with 192GB of RAM (or in my case an M5 Max MBP with 128GB of RAM). I don't know what argument you're trying to make here? You are saying this product is pointless, the point of this product is clearly stated and obvious, and now you're acting like its not relevant.
If you don't want to engage in a reasonable discussion, you can get bent.
Framework is going for a computer OEM market where they need to make margin, hence the soldered-on components and relatively limited options. People in the pre-built computer market overwhelmingly don't care about replaceable components, because anyone else will just buy components and build a PC, for both the greater market of options AND overall lower price than buying pre-built (if they don't overspec).
You literally can't make this machine without soldered components. You don't understand this computer's design goals. It's been 1.5 years since they announced the Framework Desktop. I don't get how people are still confused about this.
Mmm.. AI395 at 3.5k ( and out of stock mind ). And I still want to give it a shot when it is out. I suppose I just explained to myself the ridiculous jump in prices. The demand is crazy.
Woof. What's that going to cost? And will we actually be able to buy one, because the 128GB model has been sold out pretty much since the LLM craze started...
We can't share pricing yet, but certainly will before we open orders. The 128GB has been selling well, and we just re-opened orders with stock landing within two weeks.
It's competition is M5 Ultra, not cheap.
Rumored M5 ultra bandwidth is apparently 1100 GB/S (M3 Ultra is something like 880). This is 276 GB/S so they’re not really in the same league unfortunately. This will be considerably cheaper though.
I have a feeling the M5 ultra is going to stay in the realm of myth until RAM prices come back down to earth...
If you have to ask you don’t want to know
This machine needs better networking options to be useful, such as built-in 100Gbit QSFP28. There is not enough space between the x4 PCIe slot and the power supply for a NIC to fit, and even if there was, you'd have to cut a hole in the back of the case to get at it.
Here's me wanting to drop thousands on this so I can save a few bucks a day by not having to call cloud llms for simple distillation tasks. Lol.
It’s a fantastic dev box i got the framework desktop and its awesome as a developer desktop
So... Can it sleep with echo mem >/sys/power/state?
Abd wake up without crashing?
Tl;dr: think hard about what you’d use this much RAM for in a desktop setting and do some research about how people like the 395 for your use case. Not all use cases work well.
Anyone who thinks they are going to serve some 100+ GB LLM locally, remember that memory bandwidth becomes a key limitation for large models. While you might be able to load a model, token generation can be very slow. MoE models like Qwen3.5-122B-A10B work decently fast, but dense models of a decent size are slow and you won’t want to use them.
I’ve got a 395 system, and found that I’m quite happy with Qwen3.6-35B-A3B, generating at around 50 t/s, but the dense 27B model is 20-25 t/s and that’s the lower limit I’m willing to tolerate. So a 70b dense model is just not going to happen. That means you can’t really use that much RAM.
A reasonable use case is to have multiple smaller models loaded- you can have an image generation model loaded along with the text model. Or you can use this computer for development simultaneously with serving LLMs. Those ideas work okay. But trying to load up a single giant model is going to test your patience.
> A reasonable use case is to have multiple smaller models loaded- you can have an image generation model loaded along with the text model.
Lemonade server updated yesterday for an Arena model / ai routing.
---
The Lemonade local AI server allows easily deploying LLM workloads across Ryzen AI NPUs, Radeon GPUs, and CPUs with ease under both Windows and Linux. The most significant change with Lemonade 11.5 is the completion of the Lemonade Router that can be used for automatically routing queries to relevant models based on defined policies -- or even LLM-as-a-router for using a small LLM to in turn determine which model to route a particular request.
The Lemonade Router allows steering requests based on rule, classifier, semantic similarity, or LLM-as-router policies. The policies are defined in JSON files or can also be authored via the Lemonade GUI. More details on these router capabilities can be found via this commit.
Lemonade 11.5 also introduces a server-side job engine to let clients post multi-step recipes and managing them via /jobs endpoints. From these endpoints the muilti-step recipes can be paused / interrupted / resumed / deleted. And another one is that the lemond daemon can now act as an MCP client host to connect external stdio MCP servers.
https://www.phoronix.com/news/AMD-Lemonade-11.5
https://lemonade-server.ai/
Memory bandwidth certainly is the bottleneck on my 395+ 128GB. It's been a great server. Think I'll wait another year maybe hopefully the prices come down and there's one more gen of hardware and local llm gets even better. Excited for this!
What LLM / image / video models would this allow people to self-host that are worth it at the presumably somewhat low speed?
> Seize the means of computation
Feels like they blatantly ripped that off the title of a Cory Doctorow book [0].
[0]: https://en.wikipedia.org/wiki/The_Internet_Con
We're fans of Cory Doctorow, and he is a Framework Laptop user!
That’s a common expression.
Who blatantly ripped that off from Karl Marx...
Somewhat on-topic, if anyone here's received their Framework 13 Pro yet (which I believe began shipping this month), I'd be happy to hear a first-hand review.
Non-ECC memory.
And into the trash it goes.
Refer to https://news.ycombinator.com/item?id=48969530
the thing about ECC memory is that it doesn't support reporting of ECC errors, so it ends up just hiding the fact that your RAM has started to go bad. this is good if you replace RAM every n years, but not if you intend to use the RAM until it starts degrading.
>the thing about ECC memory is that it doesn't support reporting of ECC errors
Wait that. It does, and that's the point. Although the CPU and motherboard do of course also have to support it, but with AMD it is possible. I do it on a 9800x3d.
One bit errors are always both detected and corrected. Multibit errors are mostly at least detected.
Hear me out...
What would you say is the likelihood of an LLM hallucination vs. a correctable RAM error?
Sounds very Whataboutism.
This is about the latter, and not just the correctable ones; Detecting them also has worth.
No blog post, no order, no preorder; I think the only things about it are this sentence and a half:
> 192GB coming soon
> The most powerful Framework Desktop yet is coming soon with an AMD Ryzen™ AI Max+ PRO 495 processor and 192GB of LPDDR5X memory.
It wasn't our preferred way of making an announcement, but all that we are currently able to do this early on this pre-release processor.