tests : consolidate context state tests into test-llama-context (#30023)
* tests : consolidate the context state tests into test-llama-context test-save-load-state, test-recurrent-state-rollback and test-state-restore-fragmented check the same object from three places: the KV cache and the recurrent state of a llama_context, saved, loaded, copied, rolled back. Each had its own main, its own loop over the models, its own copy of the token and the comparison helpers, and a run of all three loaded the same models several times. They are one binary now, and the case is the unit of the test: tests/llama-context/test-case.h the interface a case implements tests/llama-context/cases-kv.cpp the cases of the KV cache tests/llama-context/cases-rs.cpp the cases of the recurrent state tests/llama-context/test-util.cpp tokens, generation, comparison tests/llama-context/test-runner.cpp the models of a case, on threads tests/llama-context/test-log.cpp the log file of a model A run is a table: a row per model, a column per case, a cell holds the verdict of that pair. The models load on several threads (-j, --threads, min(cores, 4) by default), the table is printed when it is complete and the log of a model that failed is printed after it, in the order of the rows. A case returns a verdict and the reason for it, and the runner is the only one who reports - the test does not use tests/testing.h, a run over 130 models is a matrix of verdicts and not a tree of subtests. Staging no longer travels through the environment: llama_model_set_attn_rot replaces the LLAMA_ATTN_ROT_DISABLE variable, which a model loaded in one thread used to change for the models the other threads were loading, and llama_memory_get_buffers is added for the cases that copy the buffers of a memory. common_log_set_stdo keeps the console quiet while the table is being built. ggml_aligned_malloc allocates with posix_memalign when the thread sanitizer is on. On macOS it allocates with the Mach VM, which the sanitizer does not intercept, so it never sees a page range change hands and reports the first write into recycled memory as a race - 30 of them in this test, and every one of them false. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next * cont : run single-threaded by default * cont : try to build the tests on all platforms * cont : trim down some body fat * cont : fix * cont : fix the fix
This commit is contained in:
1 parent
b476d9a762
commit
a17e94d229
27 files changed
+2772
-1965
No files matched your search
@@ -1,5 +1,11 @@
|
||||
#include "llama-memory.h"
|
||||
|
||||
#include "llama-ext.h"
|
||||
#include "llama-io.h"
|
||||
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
llama_memory_status llama_memory_status_combine(llama_memory_status s0, llama_memory_status s1) {
|
||||
bool has_update = false;
|
||||
|
||||
@@ -57,3 +63,30 @@ bool llama_memory_status_is_fail(llama_memory_status status) {
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// collects the buffers of the tensors that a state write visits, without keeping the data
|
||||
struct llama_memory_buffers_writer : public llama_io_write_i {
|
||||
std::set<ggml_backend_buffer_t> buffers;
|
||||
size_t size = 0;
|
||||
|
||||
void write(const void *, size_t n) override {
|
||||
size += n;
|
||||
}
|
||||
|
||||
void write_tensor(ggml_tensor * tensor, size_t, size_t n) override {
|
||||
buffers.insert(tensor->buffer);
|
||||
size += n;
|
||||
}
|
||||
|
||||
size_t n_bytes() override {
|
||||
return size;
|
||||
}
|
||||
};
|
||||
|
||||
std::vector<ggml_backend_buffer_t> llama_memory_get_buffers(const llama_memory_t mem) {
|
||||
llama_memory_buffers_writer writer;
|
||||
|
||||
mem->state_write(writer);
|
||||
|
||||
return std::vector<ggml_backend_buffer_t>(writer.buffers.begin(), writer.buffers.end());
|
||||
}
|
||||
Reference in new issue
Block a user