Are new OpenAI models getting better for coding?

(developer.microsoft.com)

2 points | by Zeruxe 8 hours ago ago

4 comments

  • Zeruxe 8 hours ago ago

    What even makes it worse is that they always make the models worse a few days after release, this is a wide problem across all the companies

    • bmoathn 8 hours ago ago

      do you know of anyone that's put out quantified studies on this? I'd be super curious to see. Like benchmarks at release vs 1wk vs 1mo?

      • Zeruxe 8 hours ago ago

        I would 100% be down to research it, I just know it but I cant prove it because I can feel I have agents running 24/7 almost and almost every time at the beginning of the launch they are great and they gradually decrease everyday...

  • Zeruxe 8 hours ago ago

    [flagged]