tests : consolidate context state tests into test-llama-context (#30023)

* tests : consolidate the context state tests into test-llama-context

test-save-load-state, test-recurrent-state-rollback and
test-state-restore-fragmented check the same object from three places:
the KV cache and the recurrent state of a llama_context, saved, loaded,
copied, rolled back. Each had its own main, its own loop over the
models, its own copy of the token and the comparison helpers, and a run
of all three loaded the same models several times.

They are one binary now, and the case is the unit of the test:

  tests/llama-context/test-case.h   the interface a case implements
  tests/llama-context/cases-kv.cpp  the cases of the KV cache
  tests/llama-context/cases-rs.cpp  the cases of the recurrent state
  tests/llama-context/test-util.cpp tokens, generation, comparison
  tests/llama-context/test-runner.cpp the models of a case, on threads
  tests/llama-context/test-log.cpp  the log file of a model

A run is a table: a row per model, a column per case, a cell holds the
verdict of that pair. The models load on several threads (-j, --threads,
min(cores, 4) by default), the table is printed when it is complete and
the log of a model that failed is printed after it, in the order of the
rows. A case returns a verdict and the reason for it, and the runner is
the only one who reports - the test does not use tests/testing.h, a run
over 130 models is a matrix of verdicts and not a tree of subtests.

Staging no longer travels through the environment: llama_model_set_attn_rot
replaces the LLAMA_ATTN_ROT_DISABLE variable, which a model loaded in one
thread used to change for the models the other threads were loading, and
llama_memory_get_buffers is added for the cases that copy the buffers of
a memory. common_log_set_stdo keeps the console quiet while the table is
being built.

ggml_aligned_malloc allocates with posix_memalign when the thread
sanitizer is on. On macOS it allocates with the Mach VM, which the
sanitizer does not intercept, so it never sees a page range change hands
and reports the first write into recycled memory as a race - 30 of them
in this test, and every one of them false.

Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next

* cont : run single-threaded by default

* cont : try to build the tests on all platforms

* cont : trim down some body fat

* cont : fix

* cont : fix the fix
This commit is contained in:
Georgi Gerganov authored and GitHub committed 2026-10-13 16:21:55 +03:00
1 parent b476d9a762
commit a17e94d229
27 files changed
+2772 -1965

No files matched your search

+17 -1
View File
@@ -162,6 +162,7 @@ struct common_log {
// default capacity
common_log(size_t capacity = 512) {
file = nullptr;
stdo = true;
colors = false;
prefix = false;
timestamps = false;
@@ -190,6 +191,7 @@ private:
FILE * file;
bool stdo;
bool colors;
bool prefix;
bool timestamps;
@@ -205,7 +207,9 @@ private:
bool print_entry(const common_log_entry & e) const {
if (e.is_end) return true;
e.print();
if (stdo) {
e.print();
}
if (file) {
e.print(file);
}
@@ -400,6 +404,14 @@ public:
resume();
}
void set_stdo(bool enabled) {
pause();
stdo = enabled;
resume();
}
bool get_colors() const {
return colors;
}
@@ -497,6 +509,10 @@ void common_log_set_file(struct common_log * log, const char * file) {
log->set_file(file);
}
void common_log_set_stdo(struct common_log * log, bool enabled) {
log->set_stdo(enabled);
}
void common_log_set_colors(struct common_log * log, log_colors colors) {
if (colors == LOG_COLORS_AUTO) {
log->set_colors(tty_can_use_colors());