server: slot_sched: fix lora and recurrent (#32434)

* server: fix lora and recurrent

* tests: run the hybrid SWA checkpoint test with one token per ubatch

With and without the cache, the prompt was not split into the same
ubatches, and the generated tokens are decoded one at a time while the
uncached run processes them in a batch. On CUDA this moves the top
logprobs of the tiny model by up to 0.03 and swaps near ties, so the
test failed while the cache behaved as expected.

With one token per ubatch both runs go through the same kernels and
the output is compared exactly.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Xuan-Son NguyenandPascal authored and GitHub committed 2026-11-18 00:12:20 +01:00
1 parent 3574df84b7
commit 52975342ff
8 files changed
+244 -24

No files matched your search

+3 -4
View File
@@ -306,10 +306,9 @@ bool lora_should_clear_cache(
const std::vector<common_adapter_lora_info> & current,
const std::vector<common_adapter_lora_info> & next) {
// This should always be called after determining that the two sets are
// _not_ equal. This assert is therefore some slightly wasted work and
// should be safe to remove as long as this method is called correctly.
GGML_ASSERT(!are_lora_equal(current, next));
if (are_lora_equal(current, next)) {
return false;
}
return (
!(lora_get_enabled_ids(current).empty() || lora_all_alora(current)) ||