KV Cache Eviction

Policies for discarding old key/value attention states in long sessions to manage memory.

Related terms

KV Cache (Key-Value Cache)Long‑context ModelsLatency
View on glossary index