Can LLMs Execute Algorithms?

(vkcr.dev)

3 points | by vkgcr 5 hours ago ago

1 comments

  • SamInTheShell 4 hours ago ago

    It's the training data that drives what is being observed in your post. If I trained a model to act as the interpreter you envisioned, it would likely perform better.

    They seem to be better trained at reasoning through code and building structured documents, so they will build those things better than say writing prose. Across knowledge domains, we do prose differently depending on topic and these models aren't really good at choosing what prose for what domain tailored for the human it is targeted for without explicit instructions. Even then, if it has a heavy bias towards emdashes, it'll still do the emdashes at some point regardless of your demands to stop using them in the system prompt.

    These things are great knowledge aggregators, not great programs. Depending on topic, not that great at writing.