Co-founder here—it's just me and my partner bootstrapping this thing. We've been wasting tokens left and right just trying to debug our response parsing code. Not even the AI logic, mind you, just our own sloppy stuff and don't get me started on CI: tests flaking out because GPT decides to rephrase something randomly. We got fed up paying real money to fix our bugs, so we hacked together a "Test Mode”. It routes your calls through the full pipeline (auth, logging, everything) but swaps in your sample data instead of hitting the actual provider. No tokens burned, totally deterministic, and lightning fast.
How are folks handling testing for AI integrations? Mocking always felt half-baked to us since it bypasses the real flow. What's working for you?
Co-founder here—it's just me and my partner bootstrapping this thing. We've been wasting tokens left and right just trying to debug our response parsing code. Not even the AI logic, mind you, just our own sloppy stuff and don't get me started on CI: tests flaking out because GPT decides to rephrase something randomly. We got fed up paying real money to fix our bugs, so we hacked together a "Test Mode”. It routes your calls through the full pipeline (auth, logging, everything) but swaps in your sample data instead of hitting the actual provider. No tokens burned, totally deterministic, and lightning fast.
How are folks handling testing for AI integrations? Mocking always felt half-baked to us since it bypasses the real flow. What's working for you?