Acronym
KV Cache
Key-Value Cache
Models & Training
A speed trick that stores past attention data so a model doesn't recompute it for every new token.
AJ Learning HubAJ Learning HubAcronym
Key-Value Cache
A speed trick that stores past attention data so a model doesn't recompute it for every new token.