Predictive Speculative KV Replication for Bursty LLM Inference

 (jwlabs.vercel.app)

33 points | by shreybirmiwal 10 hours ago ago

3 comments

  • holtkam2 3 hours ago ago

    Awesome animations!

  • msp26 7 hours ago ago

    the hand drawn diagrams and highlights are charming

    [-]