I was surprised to see the overall outcome was fairly consistent given different wordings during development -- but curiously the confidence levels would differ, especially in those ~50% cases.
I think it makes sense, given it's system one. Minute word differences probably make an even greater delta.
nice stick figure, though I'm not immediately sure where the trolley would go (a small animation would go a long way). Also, high-five for getting Jev access.
This is actually a pretty fun benchmark. Curious how consistent Jev is when you change the wording but keep the underlying scenario the same.
I was surprised to see the overall outcome was fairly consistent given different wordings during development -- but curiously the confidence levels would differ, especially in those ~50% cases.
I think it makes sense, given it's system one. Minute word differences probably make an even greater delta.
I learned that one human life is worth just over ten cat lives.
Jev has some preferential treatment for billionaires as well ;)
nice stick figure, though I'm not immediately sure where the trolley would go (a small animation would go a long way). Also, high-five for getting Jev access.
Thank you ;)
+1 on the animation. Thanks for the feedback.
Fortunately it's on OpenRouter for everyone now :) I think. I use my Jev BYOK, but I assume you don't need the key.