Inference is compute. If you rent an EC2 instance you pay by the hour and it has a fixed cost at the end. It's up to you to use it to the best of your abilities. Think about this is a lever you can use to say how much more concurrency you want.
We are all running lots of sessions at this point where we don't look at everything that is happening every second and if something takes an hour or an hour and 30 minutes, there is no real change. Agents can wait.
Our value added will come from more than just inference. Stay tune for more :)
not spending. You have to give something up - which is time. So you might be waiting a bit longer in certain cases and get queued but you won't get 5h/weekly/monthly limits and you won't pay by the token.
Think about the difference of paying by the minute or text messages for cell phones in the old days compared to an all you can it (with some restrictions) that you get on cell phone plans today.
Unless users violate the ToS we do not ban 24/7 and if there is more capacity at certain times you will not get queued.
In fact, the subscription gives you priority requests to get ahead of the line for certain things and the rest will flow to the background or if you have specific workloads that are not time sensitive send them there anyway.
Think about continuous AI usage of doing things like sweeping for tickets, CRM enrichment, code reviews in the background and more.
You're going to operate it at capacity, and requests will fill a queue, with the goal being, to keep it over 100% queued.
I wish you much success.
Also, you might want another pricing model, to compliment this one, for someone like a credit card processor who wants to inspect every transaction for fraud, and latency is more important, whenever you think you can compete on that metric of course.
Volume and latency versus tokenized consumption.
It does make sense. Some people want those instead of counting pennies.
LunaRoute provides inference but you pay by the concurrency and not the token. Get DeepSeek 4.1 Flash 1017, GLM 5.3 Flash, Muse Spark Open. Free trial
Does the pricing formula change the underlying cost economics ?
Inference is compute. If you rent an EC2 instance you pay by the hour and it has a fixed cost at the end. It's up to you to use it to the best of your abilities. Think about this is a lever you can use to say how much more concurrency you want.
We are all running lots of sessions at this point where we don't look at everything that is happening every second and if something takes an hour or an hour and 30 minutes, there is no real change. Agents can wait.
Our value added will come from more than just inference. Stay tune for more :)
You are saying, "the pricing structure has a spending cap limit built into it ?"
Which results in the cost structure being something like a gym membership ?
ie. oversubscribed and underutilized ?
Are you going to throttle or ban the users that fill their pipelines 24/7 ?
not spending. You have to give something up - which is time. So you might be waiting a bit longer in certain cases and get queued but you won't get 5h/weekly/monthly limits and you won't pay by the token.
Think about the difference of paying by the minute or text messages for cell phones in the old days compared to an all you can it (with some restrictions) that you get on cell phone plans today.
Unless users violate the ToS we do not ban 24/7 and if there is more capacity at certain times you will not get queued.
In fact, the subscription gives you priority requests to get ahead of the line for certain things and the rest will flow to the background or if you have specific workloads that are not time sensitive send them there anyway.
Think about continuous AI usage of doing things like sweeping for tickets, CRM enrichment, code reviews in the background and more.
Yes. I think about all these things and more.
You're going to operate it at capacity, and requests will fill a queue, with the goal being, to keep it over 100% queued.
I wish you much success.
Also, you might want another pricing model, to compliment this one, for someone like a credit card processor who wants to inspect every transaction for fraud, and latency is more important, whenever you think you can compete on that metric of course.
Volume and latency versus tokenized consumption.
It does make sense. Some people want those instead of counting pennies.
Thank you!
We do have the ability to provide dedicated servers and partial servers exactly for this type of needs as well as regulated industries and others.
We also have a specific plan with access to abliterated/uncensored models with a required KYC/KYB.
[flagged]