Together.ai: Prompt Leakage for Requests

(together.ai)

35 points | by Fe2O3 11 hours ago ago

8 comments

  • tczMUFlmoNk 9 hours ago ago

    > During development of our public GitHub repository, xorl, an AI coding agent committed a credential that provided access to the private Hugging Face repository

    My emphasis on the accountability laundering. No, "xorl" did not do this—you did. And, if "xorl" is https://github.com/togethercomputer/xorl and `togethercomputer` is together.ai, then you wrote "xorl" and are doubly responsible.

    If five years ago I had said, "During development of our public GitHub repository, my computer decided to commit a credential", this would have been immediately recognized for the embarrassing abdication of duty that it is. So, too, should this be now.

    • Dylan16807 6 hours ago ago

      You could have easily said "the company backup script" did something, and it wouldn't be embarrassing to phrase it that way. So I don't really see an issue here.

  • sans_souse 9 hours ago ago

    Am I inferring correctly here, in that Dragos basically had some sort of experiment set up and running for exactly this scenario, that effectively allowed them to trace the breach to the individual?

    • jerrythegerbil 9 hours ago ago
      • pyaamb 8 hours ago ago

        But their model wasn't

      • nsagent 8 hours ago ago

        That's a truly ingenious and scary investigation.

          Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people.
        
        The first bit of advice is on point:

          Host it yourself, if you can afford it. This is the only option where nobody else gets to read your prompts, but the hardware isn’t cheap. A Mac Studio with 512 GB of memory starts at $22,299, NVIDIA’s DGX Spark now costs $6,950, a DGX Station starts at about $105,000, and an eight-GPU B300 server lists for $570,000 or more. For most teams, this only makes sense for the most sensitive work.
        
        Despite being an NLP researcher, I never used LLMs for coding until this summer. Glad I exclusively used locally hosted models. My M5 Ultra 512GB just arrived and I'm hoping it turns out to be a wise decision for my startup.
  • ChrisArchitect 6 hours ago ago

    Related:

    Data Processing Incidents on Inference Providers

    https://news.ycombinator.com/item?id=50595554

  • nickandbro 9 hours ago ago

    The mass hacking continues…