Self-Hosted Inference for Agents

(github.com)

8 points | by rafaepta 2 days ago ago

4 comments

  • Supermancho a day ago ago

    > SIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand.

    3rd time reading this and it's still confusing.

    > an open-source inference engine that runs the models behind every agent task through one API

    "the models"? - it means "various models that are called by an agent, per task" (then lists all the tasks that it covers)

    > "It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand."

    Which says nothing about how it works.

    The About block is poorly written.

  • wyzer 2 days ago ago

    Parent deprecated?

    • rafaepta 2 days ago ago

      not really (just saw Show HN is only for one owns projects). i don't have any credits in this. Edited

      • wyzer 8 hours ago ago

        Thanks