llama : reuse graph result instances across scheduler reserves (#32089)

Allocate gf_res_reserve and both gf_res_prev entries once in the llama_context constructor. Add llm_graph_result::set_max_nodes() and reset the existing instances in sched_reserve() instead of recreating them. get_gf_res_prev() now only selects the arena for the current batch.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
This commit is contained in:
Georgi Gerganov authored and GitHub committed 2026-11-11 19:58:39 +02:00
1 parent cd2db800db
commit cb44fdb189
3 files changed
+26 -16

No files matched your search

+4
View File
@@ -1413,6 +1413,10 @@ int64_t llm_graph_result::get_max_nodes() const {
return max_nodes;
}
void llm_graph_result::set_max_nodes(int64_t max_nodes) {
this->max_nodes = max_nodes;
}
void llm_graph_result::reset() {
t_inp_tokens = nullptr;
t_inp_embd = nullptr;