How Apple fits a LLM on iPhone with surgical efficiency

(youtube.com)

9 points | by lroe 2 days ago ago

1 comments

  • paxy 2 days ago

    TL;DR

    - Quantization

    - KV caching

    - LoRA

    - Sparse MoE model

    - Dedicated AI chip

    - Offload to cloud

    Nothing really groundbreaking.