2 comments

  • Neywiny 2 hours ago ago

    What does that mean "running in real time" here? This seems like a meaningless phrase in this context. What would this look like if it wasn't running in real time?

    For example, a video decoder running in real time means it can output x seconds of output data within x seconds of wall time. But there's no time constraint on an LLM.

    • einsteinx2 2 hours ago ago

      From the thread on X it’s only 4.7 tokens/sec which personally I would find unusable for “real time” work