The idea of Tensward came to me while deploying my first fine tuned model to enterprise environment, I did most of the tests on H100, suddenly running on L4 changed everything, while working on profiling and fixing the issues, I kept thinking - why can't this be automatic? the answer was Tensward, and now it's live!
Tensward detects your hardware, ask you about your workload, get prompt examples, and then start optimising your inference, using engine analysis and other solutions that fit your use case.
The tool currently support vLLM and Llama, support for Ollama, SGLang and other serving engines are coming soon, both engines supported across NVIDIA hw and Apple Silicon
The idea of Tensward came to me while deploying my first fine tuned model to enterprise environment, I did most of the tests on H100, suddenly running on L4 changed everything, while working on profiling and fixing the issues, I kept thinking - why can't this be automatic? the answer was Tensward, and now it's live!
Tensward detects your hardware, ask you about your workload, get prompt examples, and then start optimising your inference, using engine analysis and other solutions that fit your use case.
The tool currently support vLLM and Llama, support for Ollama, SGLang and other serving engines are coming soon, both engines supported across NVIDIA hw and Apple Silicon
Great idea
Thanks! if you try it let me know how it went