> as a matter of fact, there is something really interesting about a mouse pointer feeling less like a deity floating above it all, and more like a regular in-game actor.
My counter-counter argument would be a general principle for UX designers: Are you designing a game or a tool? If you're designing a tool, don't put cutscenes in your software.
I think games are special, because their explicit purpose is to deliver an experience and often also tell a story. Within that context, I'm fine with having control restricted or yanked away if it's in service of something meaningful in the game.
The same is not true for tools (even in-game tools actually), where I want to complete some kind of task in the most efficient way possible - and often only I know the context of that task.
Unfortunately, that stuff has already seeped into UX design in a lot of forms, in particular as random "new feature" popups that usually appear at the worst possible moment and cannot be shown again. In situations like this, I'd value predictability much more than the coolness factor of a game-like UI.
I would not be happy with mouse pointer hijacking. Seems to belong in the same territory as scroll-hijacking but worse. The example case here could have been served by simply highlighting the area of interest in the UI with a red circle or a flashing pointer, whatever does the trick -- even though that may be distracting too.
There are a lot of interactions on a PC where user inputs land in the wrong place.
Claude Code and Codex in their various avatars allow us to type the next prompt while the agent is still working and responding on the earlier one. But this constantly runs into a permission prompt from running session -- either interrupting or worse entering a response to the permission prompt unintentionally.
Even during normal prompting slash commands interfere annoyingly with normal use of the slash key (i use a slash to indicate a list of two or more choices sometimes when i write).
Permission popups and confirmation dialogs that appear unexpectedly and swallow our keystrokes, spacebar and enter key hits mid sentence have always annoyed me.
Laggy devices, and resource hungry sluggish UIs compound this problem.
Software moving the mouse cursor is only acceptable when the window is full-screen. If the user makes an application go full-screen, they are opting out of the normal desktop UI conventions. It's expected that full-screen software completely takes over the UI, and there are legitimate uses for moving the mouse cursor in full-screen software, e.g. centering an invisible cursor every frame in a first-person shooter game so endless view rotation is possible. But if it's windowed then it should be impossible.
Blender (3D modeling & animation software) implements this cool thing when rotating/resizing objects: if the mouse cursor moves out of the window it reappears on the other side (enabling resizing/rotating ad infinitum).
I think a better way to implement that feature would be a mechanism for programs to temporarily enable off-screen mouse cursors. This should also track the position where the cursor would be if it had been clipped to the screen boundary as normal, and immediately return the cursor to that position when the off-screen mode ends. Note that the OS returns the cursor, not the application, so applications can't abuse this mechanism for repositioning the cursor.
I generally never want programs to go fullscreen because I like to keep taskbar shown, so I can keep track of time, notifications and whatnot.
Well designed video games that rely on fast and precise mouse input capture the cursor during the gameplay until menu is shown.
The only times I have to go fullscreen is for the games that fail to capture the cursor and where accidentally clicking outside of the game window leads to a loss.
Can't imagine a non-game program other than a video player that I would want fullscreen.
> But if it's windowed then it should be impossible.
I have one monitor, so fairly often have games/editors windowed with something else alongside them (a video, documentation, …). There are also uses where the mouse is only captured temporarily - like FPS-controls flying mode in Godot and Blender. Some image editors also allow for things like moving the cursor with arrow keys, which I find useful.
> But if it's windowed then it should be impossible.
I worked on several apps for the visually impaired that automatically move the mouse cursor to different UI elements in the front-most application, regardless of the window state. It’s a good reminder that “impossible” often just means “I haven’t accounted for that use case yet.”
If it's part of the OS's standard accessibility framework then it's acceptable. The important point is that applications shouldn't be able to arbitrarily move the mouse in situations when it's unexpected.
> The important point is that applications shouldn't be able to arbitrarily move the mouse in situations when it's unexpected.
That is quite a different statement from "It should be impossible." What should be impossible is for the OS to prevent this type of usage when it is clearly useful. Beyond accessibility, I use these features to automate testing of native macOS GUI apps.
> But to me the challenge is that it’s hard to imagine everything that needs to be preemptively captured and prohibited. I have to imagine this stuff for living, and I literally did not think anyone would just move a mouse pointer like this.
In requirements engineering, you do only codify things in the positive sense. The reason behind is that the system shall only do things that are described. If there are no things described, the system shall do nothing.
You can forbid a lot of things, but then you would have pages upon pages of requirements telling you things you may not even wanted implemented in the first place.
If any program I used moved my mouse pointer regularly, I'd quit using it. This is right up there with programs that move UI elements around or pop them up as I'm trying to interact, causing the wrong actions to occur.
I hoped for a moment this was a mouse pointer replacement that adds a tiny human character, who starts the day with energy and vigour... only to become more decrepit and worn out as 5pm approaches.
This is infuriating in certain video games, like the StarCraft II single-player campaign: you're in the middle of 4 different things, your base is being slowly nibbled on by a few run-by zerglings, need to expand, babysit workers, creep / pylons / what have you. Then the game decides to show you something. It grabs your vision cone and forces it where it wants you to look -- slowly.
This specific action -- taking the thing away from the user, while they're actively engaging with it -- should be punishable by sacrificing the designer to Satan straight up. A bit harsh but justified.
> ...but since it jumps it doesn’t lead the eyes which makes is disorienting.
What happens when you enable "mouse pointer trails"? Or is that a feature that died like a decade or two after manufacturers stopped using the extremely slow LCDs that made use of the feature all but mandatory on machines that used them?
The most intriguing case for hijacking is in Neal's "cursor camp" game linked by the author. In my own designing of applications, navigating a 2d space exceeding the boundaries of a window has a whole host of design problems. Clicking and dragging to offset your perspective is the best solution, but it is less intuitive and eliminates otherwise accessible click interactions.
Letting the view follow the pointer (therefore moving it relative to the device screen) would be remarkable for this genre of UI.
Somewhat related, but useful?: I setup AutoHotKey to, after I use alt+tab to switch to a new window, AHK will move the mouse the center of newly activated window.
Turns out it’s handy. After switching apps the cursor is in a defined location and closer to anything I want to click.
An even worse, or at least equally bad, UI fail in that Lightroom screenshot/gif: I can't even tell which buttons are supposed to be clickable and which are disabled/grayed out.
The effort put into making sure you know how to turn this feature on makes me question why it's so important to them, is the 3rd party paying them for this data?
Even major features in Adobe apps the furthest they go is those video popups rendered using webviews so they glitch into existence as a white box.
This reminds me of that old flash site with an animated rabbit chasing the mouse cursor (thinking it’s a carrot). Still trying to dig up a working link.
From the counter argument:
> as a matter of fact, there is something really interesting about a mouse pointer feeling less like a deity floating above it all, and more like a regular in-game actor.
My counter-counter argument would be a general principle for UX designers: Are you designing a game or a tool? If you're designing a tool, don't put cutscenes in your software.
I think games are special, because their explicit purpose is to deliver an experience and often also tell a story. Within that context, I'm fine with having control restricted or yanked away if it's in service of something meaningful in the game.
The same is not true for tools (even in-game tools actually), where I want to complete some kind of task in the most efficient way possible - and often only I know the context of that task.
Unfortunately, that stuff has already seeped into UX design in a lot of forms, in particular as random "new feature" popups that usually appear at the worst possible moment and cannot be shown again. In situations like this, I'd value predictability much more than the coolness factor of a game-like UI.
I would not be happy with mouse pointer hijacking. Seems to belong in the same territory as scroll-hijacking but worse. The example case here could have been served by simply highlighting the area of interest in the UI with a red circle or a flashing pointer, whatever does the trick -- even though that may be distracting too.
There are a lot of interactions on a PC where user inputs land in the wrong place.
Claude Code and Codex in their various avatars allow us to type the next prompt while the agent is still working and responding on the earlier one. But this constantly runs into a permission prompt from running session -- either interrupting or worse entering a response to the permission prompt unintentionally.
Even during normal prompting slash commands interfere annoyingly with normal use of the slash key (i use a slash to indicate a list of two or more choices sometimes when i write).
Permission popups and confirmation dialogs that appear unexpectedly and swallow our keystrokes, spacebar and enter key hits mid sentence have always annoyed me.
Laggy devices, and resource hungry sluggish UIs compound this problem.
Software moving the mouse cursor is only acceptable when the window is full-screen. If the user makes an application go full-screen, they are opting out of the normal desktop UI conventions. It's expected that full-screen software completely takes over the UI, and there are legitimate uses for moving the mouse cursor in full-screen software, e.g. centering an invisible cursor every frame in a first-person shooter game so endless view rotation is possible. But if it's windowed then it should be impossible.
Blender (3D modeling & animation software) implements this cool thing when rotating/resizing objects: if the mouse cursor moves out of the window it reappears on the other side (enabling resizing/rotating ad infinitum).
I think a better way to implement that feature would be a mechanism for programs to temporarily enable off-screen mouse cursors. This should also track the position where the cursor would be if it had been clipped to the screen boundary as normal, and immediately return the cursor to that position when the off-screen mode ends. Note that the OS returns the cursor, not the application, so applications can't abuse this mechanism for repositioning the cursor.
I generally never want programs to go fullscreen because I like to keep taskbar shown, so I can keep track of time, notifications and whatnot.
Well designed video games that rely on fast and precise mouse input capture the cursor during the gameplay until menu is shown.
The only times I have to go fullscreen is for the games that fail to capture the cursor and where accidentally clicking outside of the game window leads to a loss.
Can't imagine a non-game program other than a video player that I would want fullscreen.
> But if it's windowed then it should be impossible.
I have one monitor, so fairly often have games/editors windowed with something else alongside them (a video, documentation, …). There are also uses where the mouse is only captured temporarily - like FPS-controls flying mode in Godot and Blender. Some image editors also allow for things like moving the cursor with arrow keys, which I find useful.
> But if it's windowed then it should be impossible.
I worked on several apps for the visually impaired that automatically move the mouse cursor to different UI elements in the front-most application, regardless of the window state. It’s a good reminder that “impossible” often just means “I haven’t accounted for that use case yet.”
If it's part of the OS's standard accessibility framework then it's acceptable. The important point is that applications shouldn't be able to arbitrarily move the mouse in situations when it's unexpected.
> The important point is that applications shouldn't be able to arbitrarily move the mouse in situations when it's unexpected.
That is quite a different statement from "It should be impossible." What should be impossible is for the OS to prevent this type of usage when it is clearly useful. Beyond accessibility, I use these features to automate testing of native macOS GUI apps.
> But to me the challenge is that it’s hard to imagine everything that needs to be preemptively captured and prohibited. I have to imagine this stuff for living, and I literally did not think anyone would just move a mouse pointer like this.
In requirements engineering, you do only codify things in the positive sense. The reason behind is that the system shall only do things that are described. If there are no things described, the system shall do nothing.
You can forbid a lot of things, but then you would have pages upon pages of requirements telling you things you may not even wanted implemented in the first place.
If any program I used moved my mouse pointer regularly, I'd quit using it. This is right up there with programs that move UI elements around or pop them up as I'm trying to interact, causing the wrong actions to occur.
I hoped for a moment this was a mouse pointer replacement that adds a tiny human character, who starts the day with energy and vigour... only to become more decrepit and worn out as 5pm approaches.
I guess f.lux could be considered similar?
This is infuriating in certain video games, like the StarCraft II single-player campaign: you're in the middle of 4 different things, your base is being slowly nibbled on by a few run-by zerglings, need to expand, babysit workers, creep / pylons / what have you. Then the game decides to show you something. It grabs your vision cone and forces it where it wants you to look -- slowly.
This specific action -- taking the thing away from the user, while they're actively engaging with it -- should be punishable by sacrificing the designer to Satan straight up. A bit harsh but justified.
Windows has a “snap to default button“ setting which does the same.
Saves you a bit of movement on large screens, but since it jumps it doesn’t lead the eyes which makes is disorienting.
> ...but since it jumps it doesn’t lead the eyes which makes is disorienting.
What happens when you enable "mouse pointer trails"? Or is that a feature that died like a decade or two after manufacturers stopped using the extremely slow LCDs that made use of the feature all but mandatory on machines that used them?
There's a class in Java (Robot) that lets you move the mouse like this, I messed with that when I was younger.
The most intriguing case for hijacking is in Neal's "cursor camp" game linked by the author. In my own designing of applications, navigating a 2d space exceeding the boundaries of a window has a whole host of design problems. Clicking and dragging to offset your perspective is the best solution, but it is less intuitive and eliminates otherwise accessible click interactions.
Letting the view follow the pointer (therefore moving it relative to the device screen) would be remarkable for this genre of UI.
vim-athena would automatically move your cursor towards the command buttons whenever it made a popup appear
i thought that was genius, until i upgraded to vim-motif, which would instead move the popup to where your mouse cursor is
In the early days it was pretty common to move the pointer to the active element when one started navigating with the keyboard.
But yeah, it feels like somebody physically grabbing your hand and moving it.
The "delete someone else's pointer" in Figma is great. It would be even better if that deleted it off their screen as well.
Somewhat related, but useful?: I setup AutoHotKey to, after I use alt+tab to switch to a new window, AHK will move the mouse the center of newly activated window.
Turns out it’s handy. After switching apps the cursor is in a defined location and closer to anything I want to click.
It’s weird but it works for me.
An even worse, or at least equally bad, UI fail in that Lightroom screenshot/gif: I can't even tell which buttons are supposed to be clickable and which are disabled/grayed out.
The effort put into making sure you know how to turn this feature on makes me question why it's so important to them, is the 3rd party paying them for this data?
Even major features in Adobe apps the furthest they go is those video popups rendered using webviews so they glitch into existence as a white box.
> in the earlier days of Figma, when I prototyped an interaction where you could select someone else’s pointer and press Backspace to delete it:
Aw that looks fun!
This reminds me of that old flash site with an animated rabbit chasing the mouse cursor (thinking it’s a carrot). Still trying to dig up a working link.
There's one with a cat (and various other creatures) - oneko
https://github.com/glreno/oneko