AI optimization: 7 powerful techniques you can use today!
Both PagedAttention https://aicurated.com/ai-technologies-and-tools/ and RadixAttention enabled the possibility to cache shared prefix among multiple sequences, without duplicated copy of the prefix. This requires to compute their hidden representation of keys…
Read more »