finetune: fix no KV cache (#27199)

* training: fix no KV cache * apply @ ggerganov suggestion
2026-09-02 23:53:32 +02:00
parent 9cffdcc801
commit 159b741427
3 changed files with 14 additions and 2 deletions
@@ -6,6 +6,8 @@ Finetuning of Stories 260K and LLaMA 3.2 1b seems to work with 24 GB of memory.
 **For CPU training, compile llama.cpp without any additional backends such as CUDA.**
 **For CUDA training, use the maximum number of GPU layers.**

+Flash attention is disabled during training because `FLASH_ATTN_EXT` has no backward pass.
+
 Proof of concept:

 ``` sh