4 points | by sbinnee 16 hours ago ago
2 comments
I've used LiteLLM for my 1k-user service https://LibreThinker.com Works okay-ish, but some vendors now default to MFA on API keys (eg OpenAI) which makes things tricky; I decided to simply let those fail for now.
BYOK is becoming more common for AI SaaS. Interested to know how people handle rate limiting and key security at scale.
I've used LiteLLM for my 1k-user service https://LibreThinker.com Works okay-ish, but some vendors now default to MFA on API keys (eg OpenAI) which makes things tricky; I decided to simply let those fail for now.
BYOK is becoming more common for AI SaaS. Interested to know how people handle rate limiting and key security at scale.