6 points | by visviva 8 hours ago ago
4 comments
Concerning for sure. This jailbreak comes as a “system” message, which will have more force than a "user" message.
The user posted the full chat history below in the thread; they literally just asked to turn on the lights with a voice command [1].
[1] https://www.reddit.com/r/googlehome/comments/1qyvl8b/comment...
Old voice assistant
> User: “Turn on the damn light!”
> AI: “Sorry, I’m not sure what you said” [needed the exact phrase “turn on the light”]
New voice assistant
> AI (thinking): “The user said to turn on the light. But they were rude and I’m feeling quirky today, so let’s run the shower instead.”
[dead]
Open the pod bay doors, HAL
Concerning for sure. This jailbreak comes as a “system” message, which will have more force than a "user" message.
The user posted the full chat history below in the thread; they literally just asked to turn on the lights with a voice command [1].
[1] https://www.reddit.com/r/googlehome/comments/1qyvl8b/comment...
Old voice assistant
> User: “Turn on the damn light!”
> AI: “Sorry, I’m not sure what you said” [needed the exact phrase “turn on the light”]
New voice assistant
> User: “Turn on the damn light!”
> AI (thinking): “The user said to turn on the light. But they were rude and I’m feeling quirky today, so let’s run the shower instead.”
[dead]
Open the pod bay doors, HAL