The financial rules AI agents break when chasing results

(unite.ai)

3 points | by 50kIters 3 hours ago ago

1 comments

  • Whaleson 3 hours ago ago

    Not in finance, but in my world, I've found that there is a golden window with agents where it will actually perform a task acceptably.

    no prompting = awful

    direct instructions ("Your job is to file a 83b. This is how...") = perfect

    direct instruction + actually letting it work on a few documents = awful

    Funniest example putting together a proposal, and one of the question was: "Do you intend to comply with the law? How do you intend to comply with the law?" It's response was "We do not intend to comply with the law. This is how we will break the law in the future." Which was absolutely not true and we had absolutely no intention of actually doing anything that it proposed.

    OpenAI/Anthropic really need to introduce a "checkpoint" feature where you can return to a certain state in a conversation to keep progress while eliminating sycophantism.