Files
llama.cpp/src
Alexey Kopytko c4fb579695 moe-cache : LRFU/MQ-like hybrid cache, ClassLRU (#31233)
* moe-cache : LRFU/MQ-inspired hybrid cache

* make the clock advance by N tokens: use the real token count (ne[1] is always 1, ids have n_expert_used entries per token)

* sel is a RESHAPE node, the orig input is in src[0], so sel->src[0]->ne[1] is the count

* c for the class index, last_use for last use time

* explain the roles of these

* compute the key once

* use the key directly instead of re-reading

* show how the score is computed from the age

* remove the branch and clarify exp2 and log2 relationship

* reword the header comment and elaborate on unused slots and sorting

* not a copy, not just skip

* std::array maybe

* Revert "std::array maybe"

This reverts commit 7649b1f172db4941a854ad115bec0c7e062b65b9.

* half life it is

* add key and iteration helpers

* extract decayed_crf

* bump classes to 8, document the formula, display the cache size in token worth count

* assert the computed number of classes matches the half life
2026-11-06 11:32:54 +02:00
..
2026-10-29 15:43:33 +01:00