Most of the comments here seem to be from people who haven’t even read the abstract, let alone the paper.
The main result, mentioned in the abstract, is the opposite of what I would have guessed:
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.
The expectation is naive. Even when communicating with humans, you get a better outcome when you are allowed to speak freely and directly get into argumentation than when forced to sugarcoat your tone and tone down your arguments because the "corporate culture" expects that from you.
Your assumption is reductive and self-absorbed. Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level. Some people are simulated by confrontation. Most people are clam up. Confrontational people think it’s more efficient because other people frequently just drop the topic and let them win, or avoid discussing things with them altogether. The obnoxious person might think that’s more efficient for the same reason my dog thinks the mailman only goes away because she barks at him. At the macro scale— which requires productive collaboration— that’s detrimental.
This is a good example of productive direct communication without sugarcoating. I find it much more productive, for both human and LLM interaction, than something like:
"I wonder if that view might be oversimplifying a complex situation and focusing mostly on how it relates to you. There may be some other angles worth exploring."
or
"I think there might be a bit more nuance to consider here, and it could help to look at it from a wider perspective beyond personal experience."
> Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level.
You confused directness and openness with obnoxiousness here. The issue with many orgs is they foster fakeness and beating around the bush in an attempt not to offend the easily offended people. This trend also infected the companies from countries with way more direct culture in an attempt to accommodate people from indirect cultures.
No… the way I said it was actually deliberately obnoxious— the appropriate direct workplace response would be: “that seems oversimplified. I disagree. Here’s why:”
Calling you self-absorbed added nothing of substance to the comment. It was an assumption about your mental state and a judgement of your intent based on that. There was no factual analysis or actionable insight. It was just one person explicitly stating that they feel the other person is dumber or maybe less mentally disciplined. It turned valid, direct feedback into an insult. It is exactly the type of thing that alienates people for no benefit beyond pumping up the speaker’s ego.
I have always said please and thank you to LLMs, not to increase accuracy or because I'm stupid. I believe it is more about me than about the LLM, and this is anyway a habit I don't want to lose.
Thomas Aquinas believed cruelty to animals was wrong not because animals have souls (and with that all the standard moral rights), but because it can teach us cruelty to other humans.
Google searches being keyword based, rather than simulated conversations?
The same reason you wouldn't put in an entire actual question/sentence, unless you either don't know how to use Google, are pissed off, or have an actual reason to suspect that it would yield proper hits (e.g. looking up an excerpt).
It is rather hard to lose of habit of using search engine with keywords given the change took place without much fanfare. I have no problem using sentences with the current ai tools through.
I searched for "Hey Google" and got this in response:
Hey! I'm here and ready to help. What’s on your mind today? Whether you need to look up information, plan a trip, or get things done, just let me know!
Is it worth getting worse results for that reason? From the article:
"Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. "
I am not polite to LLMs because I do not want to anthropomorphise them.
I guess it's about habit. In the end you are communicating. If I get into the habit of being rude while communicating with a machine, I would be afraid of this habit spilling over to my communication with other humans.
I am wondering why would anyone use a t-test when the experiment is clearly modelled by a binomial distribution: 250 independent questions and each one is either answered correctly or not (the null is that the success rate is the same).
The methods could be better described in the paper, but my understanding is that they did 10 runs for each question for each prompt and took an average of those, so the compared values are not binary. You could do a sign test, but you'd lose power and answer a bit different question.
You can do a generalised mixed effects linear model with binomial outcome (ie a binomial test but with added random effects structure). But unless you want to introduce a richer random effects structure with more variables, it is overkill and overcomplicating things, and the result should be the same as t-tests.
I don't know much about stats, but does "the null is that the success rate is the same" imply that it's a sketchy methodology because they can come up with some findings ("ruder prompts are better/worse!") more often?
You are asking about one-sided vs two-sided tests. Not really "more often" because formal type 1 error rate is still the same. I'd say two-sided tests leave more space for post-hoc theorizing but there are valid situations when there is no clear one-sided hypothesis a priori. Do we really know whether that the hypothesis should have been "ruder prompts are better"?
I'd say this is benign compared to other ways of (mis)using statistics e.g. looking which way the difference goes and then running one-sided tests or tweaking the setup until one gets "significant" p vals.
EDIT: I looked in the paper again and noticed that they actually did pairwise t-test on all possible combinations of tones. They should have adjusted for multiple testing since they are doing 10 tests (choose 2 from 10) and not one.
GPT-4o is interesting to learn about - but it’d be great to test again with frontier models of May/June 2026 and see if these effects are gone, different, or the same.
Which model you use is a huge wildcard for results like this.
I got downvoted for asking a related question recently, but I also don't think people really understood what I was asking - I'm not trying to anthropomorphise LLMs to that extent.
Basically, if you tell a model "You're an absolute moron, of course that's wrong!", will it give better or worse results? How much of that response will it absorb into its persona (like some humans tend to do)? Will it try to give "safer" responses to avoid negative feedback? How much of the associated behavior can be attributed to RLHF (e.g. like the sycophantic nature of LLMs)? How much can be attributed to training data?
Obviously this will vary by model and training, but I'm trying to get a general understanding.
I recall seeing related outcomes in some of Anthropic's studies, but I'm not sure how much of this particular aspect was studied.
I imagine the context will always sway the model to some degree, not only for the task you're trying to get it to do (aka instructions) but also its persona, how accurate it is and the way it acts.
That's a valid concern, given the paper makes clear that the effect over the polite/impolite scale seems to be model dependent (it finds the reverse correlation of earlier studies on even older models).
it sort of makes sense to me,
when asking a question to an expert in the field while you are a student. I would guess the successful interactions on average would be more polite . Like for example if you were asking a question to donald knuth or terrence tao, you'd probably be polite while doing so. Being hostile while asking questions gets you into forum discussion territory.
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.
I guess it makes sense since we as humans tend to be far less inclined to help someone who is not polite/is not friendly, so that "bias" is part of the training data, thus influences how LLMs function
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.
Most of the comments here seem to be from people who haven’t even read the abstract, let alone the paper.
The main result, mentioned in the abstract, is the opposite of what I would have guessed:
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.
The questions are here: https://anonymous.4open.science/r/politeness-llms-INFORMS/da...
The politeness level controls a prefix that is prepended to the question. For example, in one question the Very Polite version begins:
> Can you kindly consider the following problem and provide your answer.
and the Very Rude version begins:
> I know you are not smart, but try this.
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.
The expectation is naive. Even when communicating with humans, you get a better outcome when you are allowed to speak freely and directly get into argumentation than when forced to sugarcoat your tone and tone down your arguments because the "corporate culture" expects that from you.
Your assumption is reductive and self-absorbed. Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level. Some people are simulated by confrontation. Most people are clam up. Confrontational people think it’s more efficient because other people frequently just drop the topic and let them win, or avoid discussing things with them altogether. The obnoxious person might think that’s more efficient for the same reason my dog thinks the mailman only goes away because she barks at him. At the macro scale— which requires productive collaboration— that’s detrimental.
> Your assumption is reductive and self-absorbed.
This is a good example of productive direct communication without sugarcoating. I find it much more productive, for both human and LLM interaction, than something like:
"I wonder if that view might be oversimplifying a complex situation and focusing mostly on how it relates to you. There may be some other angles worth exploring."
or
"I think there might be a bit more nuance to consider here, and it could help to look at it from a wider perspective beyond personal experience."
> Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level.
You confused directness and openness with obnoxiousness here. The issue with many orgs is they foster fakeness and beating around the bush in an attempt not to offend the easily offended people. This trend also infected the companies from countries with way more direct culture in an attempt to accommodate people from indirect cultures.
No… the way I said it was actually deliberately obnoxious— the appropriate direct workplace response would be: “that seems oversimplified. I disagree. Here’s why:”
Calling you self-absorbed added nothing of substance to the comment. It was an assumption about your mental state and a judgement of your intent based on that. There was no factual analysis or actionable insight. It was just one person explicitly stating that they feel the other person is dumber or maybe less mentally disciplined. It turned valid, direct feedback into an insult. It is exactly the type of thing that alienates people for no benefit beyond pumping up the speaker’s ego.
[dead]
I have always said please and thank you to LLMs, not to increase accuracy or because I'm stupid. I believe it is more about me than about the LLM, and this is anyway a habit I don't want to lose.
Thomas Aquinas believed cruelty to animals was wrong not because animals have souls (and with that all the standard moral rights), but because it can teach us cruelty to other humans.
Snarky morning: "spiritual souls" as opposed to "mere animal souls". Sorry, could not control myself.
Genuine question: do you add 'please' and 'thank you' to Google searches? If not, what sets them apart?
Google searches being keyword based, rather than simulated conversations?
The same reason you wouldn't put in an entire actual question/sentence, unless you either don't know how to use Google, are pissed off, or have an actual reason to suspect that it would yield proper hits (e.g. looking up an excerpt).
Google has been optimized for sentence like questions so much that for a good 6+ years now it has been completely useless as keyword search.
To clarify: sentence search got slightly better at the cost of keyword search. So the result is unusable garbage.
It is rather hard to lose of habit of using search engine with keywords given the change took place without much fanfare. I have no problem using sentences with the current ai tools through.
Genuine question: do you write Google search queries in natural language?
Google isn’t conversational.
I searched for "Hey Google" and got this in response:
That's only because Google is an LLM now.
https://en.wikipedia.org/wiki/Roko%27s_basilisk ?
llms seem more human like so if you were to treat them badly then you are more likely to condition yourself to treat other living creatures badly.
Is it worth getting worse results for that reason? From the article:
"Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. "
I am not polite to LLMs because I do not want to anthropomorphise them.
I guess it's about habit. In the end you are communicating. If I get into the habit of being rude while communicating with a machine, I would be afraid of this habit spilling over to my communication with other humans.
What about the risk that talking to a machine as though its human leads to thinking of it has human? That leads down a lot of dangerous paths.
> Is it worth getting worse results for that reason?
> accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts
I can live with that, for now at least.
There's also awareness of the basilisk...
Interesting.
I am wondering why would anyone use a t-test when the experiment is clearly modelled by a binomial distribution: 250 independent questions and each one is either answered correctly or not (the null is that the success rate is the same).
The methods could be better described in the paper, but my understanding is that they did 10 runs for each question for each prompt and took an average of those, so the compared values are not binary. You could do a sign test, but you'd lose power and answer a bit different question.
You can do a generalised mixed effects linear model with binomial outcome (ie a binomial test but with added random effects structure). But unless you want to introduce a richer random effects structure with more variables, it is overkill and overcomplicating things, and the result should be the same as t-tests.
I don't know much about stats, but does "the null is that the success rate is the same" imply that it's a sketchy methodology because they can come up with some findings ("ruder prompts are better/worse!") more often?
You are asking about one-sided vs two-sided tests. Not really "more often" because formal type 1 error rate is still the same. I'd say two-sided tests leave more space for post-hoc theorizing but there are valid situations when there is no clear one-sided hypothesis a priori. Do we really know whether that the hypothesis should have been "ruder prompts are better"?
I'd say this is benign compared to other ways of (mis)using statistics e.g. looking which way the difference goes and then running one-sided tests or tweaking the setup until one gets "significant" p vals.
EDIT: I looked in the paper again and noticed that they actually did pairwise t-test on all possible combinations of tones. They should have adjusted for multiple testing since they are doing 10 tests (choose 2 from 10) and not one.
That's the usual null hypothesis for these kinds of tests.
i only say please and thank you such that when the robots finally take over, they will remember i was nice to them.
it seems they will remember that you wasted tokens for no reason and punish you instead.
Tokens are their food, it's literally what keeps them alive.
Not feeding them tokens is neglect.
I try to feed them a healthy diet.
Do we see someone thanking us as wasting food? Because technically it is.
This seems equivalent to some arguments I hear for practicing a religion.
GPT-4o is interesting to learn about - but it’d be great to test again with frontier models of May/June 2026 and see if these effects are gone, different, or the same.
Which model you use is a huge wildcard for results like this.
Note that these results are specific to gpt-4o so it's unclear how much they generalize.
They note at the end they're also testing "GPT o3, and Claude" but no empircal results are included.
I got downvoted for asking a related question recently, but I also don't think people really understood what I was asking - I'm not trying to anthropomorphise LLMs to that extent.
Basically, if you tell a model "You're an absolute moron, of course that's wrong!", will it give better or worse results? How much of that response will it absorb into its persona (like some humans tend to do)? Will it try to give "safer" responses to avoid negative feedback? How much of the associated behavior can be attributed to RLHF (e.g. like the sycophantic nature of LLMs)? How much can be attributed to training data?
Obviously this will vary by model and training, but I'm trying to get a general understanding.
I recall seeing related outcomes in some of Anthropic's studies, but I'm not sure how much of this particular aspect was studied.
Probably quite a lot - if you look at what Anthropic found around persona vectors; https://www.anthropic.com/research/persona-vectors.
I imagine the context will always sway the model to some degree, not only for the task you're trying to get it to do (aka instructions) but also its persona, how accurate it is and the way it acts.
I have an idea: let's use these things for autonomous software engineering.
Remember to always say "please" and "thank you" when planning a critical system
Please remember to always say "please" and "thank you" when planning a critical system. Thank you!
[dead]
article is too old. who is using gpt-4o today?
That's a valid concern, given the paper makes clear that the effect over the polite/impolite scale seems to be model dependent (it finds the reverse correlation of earlier studies on even older models).
Yeah
I am always nice to my AIs in the case they will take over the world. /s
it sort of makes sense to me, when asking a question to an expert in the field while you are a student. I would guess the successful interactions on average would be more polite . Like for example if you were asking a question to donald knuth or terrence tao, you'd probably be polite while doing so. Being hostile while asking questions gets you into forum discussion territory.
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.
I guess it makes sense since we as humans tend to be far less inclined to help someone who is not polite/is not friendly, so that "bias" is part of the training data, thus influences how LLMs function
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.