JevBench: Benchmarking decision models (Clef, pplx-decider and 15 more)

(leanmcp.github.io)

3 points | by paidev 4 hours ago ago

2 comments

  • verdverm 3 hours ago ago

    Thoughts on adding benchmarks like coin flipping, dice rolls, and other well known outcome questions?

    Some of the decision models do really poorly on this with the outcome probability and being very confident about the decision.

  • paidev 4 hours ago ago

    [dead]