How Apple fits a LLM on iPhone with surgical efficiency

 (youtube.com)

9 points | by lroe 16 hours ago ago

1 comments

  • paxy 13 hours ago ago

    TL;DR

    - Quantization

    - KV caching

    - LoRA

    - Sparse MoE model

    - Dedicated AI chip

    - Offload to cloud

    Nothing really groundbreaking.