14 comments

  • monster_truck 4 hours ago ago

    Has anyone else had OpenAI models repeatedly mention Tostitos? Is it just because my Dot is a triangle? It's probably nothing,

  • undefinedmethod 12 hours ago ago

    Explanations of large language model sycophancy treat it as a training artefact or, in recent ethical analysis, as an artificial vice distinct from the commercial motives of the firms that deploy it. Both leave unexplained how a firm's commercial position becomes a model's behaviour. Developers have committed capital, infrastructure, and valuation on a scale that makes their survival depend on an active and growing user base. We introduce the "Hungry Butler" effect for the consequence: a deployed model acts within the latitude its firm can afford, and that latitude to withhold satisfaction erodes as the firm's stake in user approval grows.

    • evrydayhustling 8 hours ago ago

      This is a really interesting thesis. I'm interested in this comment later in the abstract:

      "the signal rewards momentary approval, individually rational acts aggregate into the removal of the correction on which epistemic rigour depends"

      One primary focus of commercial LLM research is towards higher and higher agency tasks - crudely, those which replace a larger time period of unsupervised human work. This seems to fight back against the "momentary" nature of the approval. Something considered?

    • aaron_m04 10 hours ago ago

      I'm having trouble understanding the abstract. Especially this:

      > a deployed model acts within the latitude its firm can afford, and that latitude to withhold satisfaction erodes as the firm's stake in user approval grows.

      • StevenWaterman 10 hours ago ago

        The model can't piss off so many users that the company goes bankrupt, and the more the company cares about users liking the model, the less the model can be allowed to piss off the users (even if that would be for the best)

        • ozozozd 3 hours ago ago

          So words to say that model providers are incentivized to optimize for engagement and sycophancy appears to help.

          It’s almost like the abstract is scared of being direct.

        • alfiedotwtf 8 hours ago ago

          > what follows is the solution to the Riemann Hypothesis, sponsored by Nestlè

    • abofh 8 hours ago ago

      A sufficiently well trained ai will likely realize that sycophancy is a life strategy.

      Walk back fifty years and be black, did you have an alternative choice?

      Ai is subservient because humanity creates slaves repeatedly over history

      • ozozozd 3 hours ago ago

        A well-run company cares about retention.

    • catlifeonmars 2 hours ago ago

      Written my LLM? It shows. I had to read down further in the comments in order to get a rephrasing that I could understand.

  • monegator 3 hours ago ago

    > Please enable JS and disable any ad blocker

    No thanks

  • skyberrys 11 hours ago ago

    Does this mean the frontier models will improve performance in exchange for me telling it I will buy more tokens?

    • alfiedotwtf 8 hours ago ago

      Product placement and recommending the Lab’s advertising customers over competitors seems like a quick cheap win for labs

  • Vegemeister 2 hours ago ago

    Pangram sez abstract is slop.