The most interesting part about this for me is that they decided to create their own language or DSL for the task at hand. So it's not just a large language model; it's an LLM with its own language.
I have a feeling that the best AI systems to come will, in fact, be a complete package like this: a harness, a DSL, and an entire package designed to produce certain outcomes cheaper and faster.
And producing that complete package is why software engineering will not be obsolete.
I agree. I'm waiting for someone to invent a programming language designed for LLMs where for a given partial program p and candidate token t it's possible to tell whether p+t can be the prefix of a correct program or not so that t can be excluded from the LLM's probability distribution at generation time, so the LLM can only generate correct programs. Or something like that.
No, practical programs don't need full Turing completeness. Most programs you want to write you intend to either halt or run indefinitely, because you want to avoid crashes.
There is a well known problem with LLMs that if you feed it its own output, it gradually gets worse and worse.
We haven't really understood what the limitations of LLMs are. And I do not know. But as a person who uses Fable and Sol regularly to design my own new programming language, they suck at the task of defining new systems coherently. So far no AI I have tried is good at defining new coherent systems well. I suspect it's because of the recursive problem of using it on its own output.
So, a human is still needed to define and write the sort of first genesis of the system, and then AI can take it once the problem has been defined. But defining the problem and using it on itself is what AI is really bad at (for now).
I think there is a big misunderstanding in the space around what Gen UI is and what its used for. Lots of folks refer to it as a framework for building web apps - its not. Gen UI is a DSL for LLM to build UIs on the fly in a multi turn converstation - those are - throw away, one off interfaces or visualization. The reason for the DSL is pragmatism - standardisation and token savings.
The html/css/js or a react app built by an LLM is not Gen UI.
Oh... amazing. just had a vision of being able to be in a meeting and talk through an User Interface design / review, while in a zoom meeting or whatever.
...i like.
---
- Design system / Component lib
- Live view of what components, tokens, other things... on the left side of the screen.
- You're in the meeting and talking while talking and transcribing and doing the full duplex voice. You say, "Find what tables and customizations we have available" and the list starts to filter to tables and customizations.
- "Let's add that table to the page; left side; 3/4 width of page. Headers should be static for vertical scroll, ..."
- The table is added to the page.
- "Nah, i don't like it. Let's change that table component to have larger headers..."
yes, i like--let's see what Astra Pro pops out with.
The problem with that is that it only works with simple, least interactive UIs. Each new UI a human will be presented needs to be learned to be ised effectively otherwise a user will be lost.
Having said that, imo, Gen UI only makes sens as a presentation layer - not controls. Unless LLM will be using a set of very well defined and homogenic components like table, forms, small widgets.
> The problem with that is that it only works with simple, least interactive UIs.
I believe that's incorrect. you can define the architecture to be able to provide data based on information from the frontend and the components just need to define a query and data structure or something -- will have a prototype soon
I've been thinking for a while that something like this could be a solution for the suboptimal UX in the digital assets space.
Imagine a very small LLM embedded into a MetaMask equivalent where you just specify the task you want to do - ie, "I want to send USDT on mainnet", "I want to import the token at address 0x..." - and the wallet assembles the UX for this task for the human to execute.
Pre-LLMs Steve Krug wrote "Don't make me think"
Now we come to a generation of random UIs that will confuse the life out of users and, being non deterministic, be a nightmare for support teams; though they'll probably have no real support, just more llms.
It should be possible to run on Mac via https://github.com/mmastrac/diffgemma, but I'm at rustconf right now and I can't download weights on hotel wifi easily.
Quite interesting to see no real comments here for 50+ minutes, so I will kick it off.
I'm a huge believer in this future of software. Having the UI layer completely abstracted from pre-written code and dynamically generated "on the fly" based on the context of the user is, I believe, the future.
Just think about the complexity of localization and how many IFs you had to write to solve different language versions, etc. in the old PHP code. A lot of that complexity can simply disappear.
Having dynamically built UI won't only be better for the user experience, it can actually allow us to create much more personalized experiences (I hate when UI teams constantly redesign perfectly fine software).
Interestingly, this will open up a completely new consumption interface, because I believe there will be a UI predefined by the creator of the application (your day 1 user experience) that will then evolve into a more personalized experience over time.
I agree somewhat. An example might be a .md file describing a UI for commonly used tool that is invoked whenever you reference it. This could be a stripped down version of a complex UI for some software that has a lot of different uses (like 3D modeling programs and image editors) allowing the user to focus on the subset of work they do with it.
> Just think about the complexity of localization and how many IFs you had to write to solve different language versions, etc. in the old PHP code. A lot of that complexity can simply disappear.
IFs! Oh no! Throwing all of this into a non-deterministic and expensive black box is making it less complex, you say?
Great link, I haven't seen that before. I saw someone at Microsoft make an "OS" that was just copilot chats per-window, generating the HTML. I recreated it and it's not great, but it's a fun toy that _feels_ transformative, unlike almost every AI product ever made besides the fundamental chat interface.
Imagine AppLess running a model as good as qwen at 20,000 tok/sec. It would be generated in a shorter amount of time as downloading a webpage right now. If this works out the consequences are kind of scary. The end of SaaS, the end of software being the moat or the property of companies, the embolstering of data protection (since that's fundamentally what code operates)...
> I hate when UI teams constantly redesign perfectly fine software
... with ...
> Having the UI layer completely abstracted from pre-written code and dynamically generated "on the fly" based on the context of the user is, I believe, the future
In this scenario there's still no guarantee that the UI won't randomly change. There's no guarantee that the ui generated for the user will be the same visit to visit.
Why? Is two seconds too long? Would your other constraints be easier (usability and hardware spec) if this was longer? Does anyone actually need a UI generated in under a second?
The term "Generative UI" refers to a front-end design approach where an AI model dynamically builds a UI in real time instead of relying on static, hard-coded templates.
Similar to how AI assitants give you answer directly instead of making you search through walls of text. Generative UI builds the right interface based on the context and intent of the user.
In Generative UI, the interface needs to built in realtime based on context and intent of the user. Hence the constraints. Ideally we are targeting sub 500ms to compete with current software.
Gen UI is meant to be design agnostic, the output is just the content and the form. It is on the implementation, agentic or human to make it look good.
There's an image in the article and a full website with more media is just 1 click away. Instead you resorted to typing 272 characters not including ENTER, and I doubt that was easier than clicking the logo to visit the homepage.
Typing this comment also did not solve your problem, because that would require the author to read your comment, add more screenshots and it would require that you revisit it.
When the blog title is "world's first model for Generative UI", you are supposed to show something that GPT/Claude couldn't do. Instead it's all benchmark.
I don't know what media you are talking about. It's all slop worse than current slop.
GenUI isn't about designing cosmetic "skins." (Usually, anyway. I guess it could be used for that)
It's generally for letting users customize the own workflows. How many times have you, or one of your users, liked a piece of software because it mostly fits an existing workflow but that remaining 20% is an annoyance, or maybe even a dealbreaker?
This is probably more common for businesses. They have existing procedures. and they want your software to fit into their existing processes and workflows... not the other way around.
GenUI is far from a one size fits all approach or magic bullet, but it can address a lot of those situations that either would have been dealbreakers, annoyances, or change requests. I suppose it can also help with user retention; once they've put the time and effort into customizing your product they theoretically are less likely to switch to a competitor.
Existing OpenAI/Anthropic models seem to already handle this pretty well. As you might expect, letting users describe their own UI is pretty easy. The hard part is making it work and making sure they don't escape their sandbox...
As a user, most GUIs are awful. I’m fairly certain that this thing could, for example, vibe up a better UI for Amazon Music in less time than it takes me to find the music I’ve purchased and downloaded (because the system is more interested in funneling me toward a streaming subscription that I don’t have).
Of course pushing users toward subscription services they don’t need is part of the design goal. So I guess something where the user vibecodes up their own UI will not become standard. But we can dream.
Another way to think of this is you just ship the UI DSL, and the user can get the app to customize it themselves with built-in guardrails. Everything should be LCARS at this point.
Nothing stops you from caching known states and workflows, or simply making the "fast" part of your interface fixed. I think this could be genuinely useful for one-off cases for which no interface exists, or simply for interface prototyping and design.
The goal of generative UI should be to make software more personal while preserving the workflows people already know. Two users might have very different interfaces, but each should have a consistent experience over time.
I can only imagine the troubleshooting and customer support experience. Yet another problem created by “ai” that’s probably only solvable with more “ai”
I understand the skepticism. We hear this often and are working on it. As AI agents become more common in SaaS, these experiences will become more reliable and ready for everyday use.
Chat Ui killed the GUI star. language is The ultimate UI. I guess you still need graphs (mainly so you can have dramatic moments in movies), but that's it
The most interesting part about this for me is that they decided to create their own language or DSL for the task at hand. So it's not just a large language model; it's an LLM with its own language.
I have a feeling that the best AI systems to come will, in fact, be a complete package like this: a harness, a DSL, and an entire package designed to produce certain outcomes cheaper and faster.
And producing that complete package is why software engineering will not be obsolete.
I agree. I'm waiting for someone to invent a programming language designed for LLMs where for a given partial program p and candidate token t it's possible to tell whether p+t can be the prefix of a correct program or not so that t can be excluded from the LLM's probability distribution at generation time, so the LLM can only generate correct programs. Or something like that.
doesn't this imply it would solve the halting problem?
No, practical programs don't need full Turing completeness. Most programs you want to write you intend to either halt or run indefinitely, because you want to avoid crashes.
Why can't an AI be trained to generate "the whole package"?
It too is just software
There is a well known problem with LLMs that if you feed it its own output, it gradually gets worse and worse.
We haven't really understood what the limitations of LLMs are. And I do not know. But as a person who uses Fable and Sol regularly to design my own new programming language, they suck at the task of defining new systems coherently. So far no AI I have tried is good at defining new coherent systems well. I suspect it's because of the recursive problem of using it on its own output.
So, a human is still needed to define and write the sort of first genesis of the system, and then AI can take it once the problem has been defined. But defining the problem and using it on itself is what AI is really bad at (for now).
Remember when everything was iSomething? The original iPod and iPhone created a real trend back then.
Seems like many AI products ride the same wave now. Open WebUI, OpenHands, OpenUI. I am a bit more dubious about this affiliation, though.
I think there is a big misunderstanding in the space around what Gen UI is and what its used for. Lots of folks refer to it as a framework for building web apps - its not. Gen UI is a DSL for LLM to build UIs on the fly in a multi turn converstation - those are - throw away, one off interfaces or visualization. The reason for the DSL is pragmatism - standardisation and token savings.
The html/css/js or a react app built by an LLM is not Gen UI.
Oh... amazing. just had a vision of being able to be in a meeting and talk through an User Interface design / review, while in a zoom meeting or whatever.
...i like.
---
- Design system / Component lib
- Live view of what components, tokens, other things... on the left side of the screen.
- You're in the meeting and talking while talking and transcribing and doing the full duplex voice. You say, "Find what tables and customizations we have available" and the list starts to filter to tables and customizations.
- "Let's add that table to the page; left side; 3/4 width of page. Headers should be static for vertical scroll, ..."
- The table is added to the page.
- "Nah, i don't like it. Let's change that table component to have larger headers..."
yes, i like--let's see what Astra Pro pops out with.
The problem with that is that it only works with simple, least interactive UIs. Each new UI a human will be presented needs to be learned to be ised effectively otherwise a user will be lost.
Having said that, imo, Gen UI only makes sens as a presentation layer - not controls. Unless LLM will be using a set of very well defined and homogenic components like table, forms, small widgets.
> The problem with that is that it only works with simple, least interactive UIs.
I believe that's incorrect. you can define the architecture to be able to provide data based on information from the frontend and the components just need to define a query and data structure or something -- will have a prototype soon
OpenAI, OpenAPI and now OpenUI. Explain this to a non tech person...
I actually thought this was OpenAI until I read your comment
My browser popped up a modal saying 'Did you mean openai.com? Attackers sometimes mimic sites by making hard-to-see changes to the web address'.
I've been thinking for a while that something like this could be a solution for the suboptimal UX in the digital assets space.
Imagine a very small LLM embedded into a MetaMask equivalent where you just specify the task you want to do - ie, "I want to send USDT on mainnet", "I want to import the token at address 0x..." - and the wallet assembles the UX for this task for the human to execute.
yep, this is usecases we are trying to solve. Current model fits in consumer hardware. Hopefully we can fit this into a phone soon.
Pre-LLMs Steve Krug wrote "Don't make me think" Now we come to a generation of random UIs that will confuse the life out of users and, being non deterministic, be a nightmare for support teams; though they'll probably have no real support, just more llms.
It should be possible to run on Mac via https://github.com/mmastrac/diffgemma, but I'm at rustconf right now and I can't download weights on hotel wifi easily.
let me try this out as well
https://github.com/mmastrac/diffgemma#custom-quantization has some instructions on (naively) quantizing the upstream model as well.
Quite interesting to see no real comments here for 50+ minutes, so I will kick it off.
I'm a huge believer in this future of software. Having the UI layer completely abstracted from pre-written code and dynamically generated "on the fly" based on the context of the user is, I believe, the future.
Just think about the complexity of localization and how many IFs you had to write to solve different language versions, etc. in the old PHP code. A lot of that complexity can simply disappear.
Having dynamically built UI won't only be better for the user experience, it can actually allow us to create much more personalized experiences (I hate when UI teams constantly redesign perfectly fine software).
Interestingly, this will open up a completely new consumption interface, because I believe there will be a UI predefined by the creator of the application (your day 1 user experience) that will then evolve into a more personalized experience over time.
So much room to grow in this space.
I agree somewhat. An example might be a .md file describing a UI for commonly used tool that is invoked whenever you reference it. This could be a stripped down version of a complex UI for some software that has a lot of different uses (like 3D modeling programs and image editors) allowing the user to focus on the subset of work they do with it.
> Just think about the complexity of localization and how many IFs you had to write to solve different language versions, etc. in the old PHP code. A lot of that complexity can simply disappear.
IFs! Oh no! Throwing all of this into a non-deterministic and expensive black box is making it less complex, you say?
this exactly the future we are working towards as well. You should checkout AppLess http://github.com/thesysdev/appless
Great link, I haven't seen that before. I saw someone at Microsoft make an "OS" that was just copilot chats per-window, generating the HTML. I recreated it and it's not great, but it's a fun toy that _feels_ transformative, unlike almost every AI product ever made besides the fundamental chat interface.
To take this a step further: https://chatjimmy.ai/
Imagine AppLess running a model as good as qwen at 20,000 tok/sec. It would be generated in a shorter amount of time as downloading a webpage right now. If this works out the consequences are kind of scary. The end of SaaS, the end of software being the moat or the property of companies, the embolstering of data protection (since that's fundamentally what code operates)...
Exciting times indeed
How do you reconcile ...
> I hate when UI teams constantly redesign perfectly fine software
... with ...
> Having the UI layer completely abstracted from pre-written code and dynamically generated "on the fly" based on the context of the user is, I believe, the future
In this scenario there's still no guarantee that the UI won't randomly change. There's no guarantee that the ui generated for the user will be the same visit to visit.
> Interfaces must be generated in under a second
Why? Is two seconds too long? Would your other constraints be easier (usability and hardware spec) if this was longer? Does anyone actually need a UI generated in under a second?
The term "Generative UI" refers to a front-end design approach where an AI model dynamically builds a UI in real time instead of relying on static, hard-coded templates.
This just seems incredibly wasteful. What value does that add?
Similar to how AI assitants give you answer directly instead of making you search through walls of text. Generative UI builds the right interface based on the context and intent of the user.
It keeps the users on their toes.
In Generative UI, the interface needs to built in realtime based on context and intent of the user. Hence the constraints. Ideally we are targeting sub 500ms to compete with current software.
I get annoyed if a webpage takes longer than 1-2 seconds to generate right now. You don't?
[dead]
Generative UI is unsolved because current models do not have taste and end up generating the same kind of slop.
I'm amazed that this blog doesn't even have a single screenshot/photo of the kind of UI they can generate.
Focusing on benchmarks in this domain feels very wrong.
Gen UI is meant to be design agnostic, the output is just the content and the form. It is on the implementation, agentic or human to make it look good.
design is the hard problem. i don't know what hard problem this is trying to solve here
There's an image in the article and a full website with more media is just 1 click away. Instead you resorted to typing 272 characters not including ENTER, and I doubt that was easier than clicking the logo to visit the homepage.
Typing this comment also did not solve your problem, because that would require the author to read your comment, add more screenshots and it would require that you revisit it.
When the blog title is "world's first model for Generative UI", you are supposed to show something that GPT/Claude couldn't do. Instead it's all benchmark.
I don't know what media you are talking about. It's all slop worse than current slop.
Oui ma chérie
Someone steel man the case for users actually wanting to be a UI designer for the application they pay you for.
GenUI isn't about designing cosmetic "skins." (Usually, anyway. I guess it could be used for that)
It's generally for letting users customize the own workflows. How many times have you, or one of your users, liked a piece of software because it mostly fits an existing workflow but that remaining 20% is an annoyance, or maybe even a dealbreaker?
This is probably more common for businesses. They have existing procedures. and they want your software to fit into their existing processes and workflows... not the other way around.
GenUI is far from a one size fits all approach or magic bullet, but it can address a lot of those situations that either would have been dealbreakers, annoyances, or change requests. I suppose it can also help with user retention; once they've put the time and effort into customizing your product they theoretically are less likely to switch to a competitor.
Existing OpenAI/Anthropic models seem to already handle this pretty well. As you might expect, letting users describe their own UI is pretty easy. The hard part is making it work and making sure they don't escape their sandbox...
agreed. the core idea behind Generative UI is personalisation
As a user, most GUIs are awful. I’m fairly certain that this thing could, for example, vibe up a better UI for Amazon Music in less time than it takes me to find the music I’ve purchased and downloaded (because the system is more interested in funneling me toward a streaming subscription that I don’t have).
Of course pushing users toward subscription services they don’t need is part of the design goal. So I guess something where the user vibecodes up their own UI will not become standard. But we can dream.
Please for the love of god have a human write something that you expect other humans to read
"The gap became the north star" is written large enough though: I could read it with my own Aieyes!
It's also the only sentence I read on that page before closing it, of course.
Oh no.
oh, you eye?
i prefer wee
Do you hate the designer changed the UI of your favorite app so you have to relearn it every three months?
Congrats! Now you need to relearn it every time you open the app.
Another way to think of this is you just ship the UI DSL, and the user can get the app to customize it themselves with built-in guardrails. Everything should be LCARS at this point.
It practically is, considering that "looks good on TV" seems to be a much higher priority than "usable".
I can think of one use case for this: Dashboards to answer one-off questions. Think PowerBI, Tableau, etc.
yes, conversational analytics is what most of users use OpenUI for.
Nothing stops you from caching known states and workflows, or simply making the "fast" part of your interface fixed. I think this could be genuinely useful for one-off cases for which no interface exists, or simply for interface prototyping and design.
The goal of generative UI should be to make software more personal while preserving the workflows people already know. Two users might have very different interfaces, but each should have a consistent experience over time.
I can only imagine the troubleshooting and customer support experience. Yet another problem created by “ai” that’s probably only solvable with more “ai”
I understand the skepticism. We hear this often and are working on it. As AI agents become more common in SaaS, these experiences will become more reliable and ready for everyday use.
> The goal of generative UI should be to make software more personal while preserving the workflows people already know.
These are inherently contradictory statements.
When people actually did research instead of vibe-coding, they learned it the hard way.
Why will you be using UI when you have an agent? UI is text or voice input. A website just has to look pretty
Chat Ui killed the GUI star. language is The ultimate UI. I guess you still need graphs (mainly so you can have dramatic moments in movies), but that's it
love that song! Not every interaction would be conversational, that's why we spent more than 50 years building GUI
That's why I have my own model restyle the delivered UI to my preferred UI.
You missed the point, it's not for you to write.
[dead]