ggml : add backend scheduler tests (#27258)

* Adds initial scheduler stress tests

* Adds new sleep op to simplify scheduler testing

* Adds testcase addressing https://github.com/ggml-org/llama.cpp/issues/23321

* Adds experimental vulkan GGML_OP_SLEEP

* Improves structure and naming of previous tests. Adds new tests,
including one which detects the nkvo-bug correctly, and validates the
proposed hotfix.

* Cleanup: Removes CUDA specific comments and nvtx-markers

* Fix: forgot to add sleep specific test case

* Fix oversights identified by CI

* Drops default prints so that it behaves like other ctests. Adds -v flag
to toggle this behavior

* Fix: Early exit in CPU-only settings (cause for graviton CI fails)

* Improves stress_test_linked_list by removing overlap with
test_chain_all_backends and test_pair_user_inputs. The latter ones test
either activations or user inputs in isolation and just once, whereas
the stress test tests all combinations multiple rounds

* Exclude ACCEL devices from new tests due to incompatibility

* Fixes vulkan test failure in llvmpipe

* Adds asserts, improves comments and some checks

* Fixes nitpicks in white-space and variable naming

* Adds comments that certain vk drivers could time OP_SLEEP incorrectly

* Drops limitiation to CPU/GPU in test-sleep

* Aligns OP_SLEEP guards in cuda with vk guards (input shape and
contiguity)

* Implements feedback for OP_SLEEP to work in place (no copy)

* Apply non-code suggestions  from JohannesGaesslers code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* Removes struct backend_supports; enables tests to return skip, instead
of only pass/fail

* Make clang-tidy happy

* Removes sched_backend_caps, improves create_test_scheduler according to
feedback (L149)

* Removes "name" field from struct sched_check in favor of
ggml_set_name/ggml_get_name

* Drops unnecessary shader_clock.comp test and guards

* Replaces sched_ prefixes with helper_ prefix

* Moves test suite to new RAII types for better memory management

* Adds sleep option to test_chain_all_backends

* Adds sleep option to first-run arm of the y-shaped test

* Fix switchcase order for legibility in stress_test_dag

* Simplfies stress_test_linked_list via double for-loop

* Adds pseudo-random input to stress_test_linked_list, and runs all
transitions both with and without CPU input (next to activation copy as
input)

* Fix RPC Proto patch due to GGML_OPS bump

* Adds ggml-cpp.h treatment to test-sleep.cpp

* Adds TODO pointers for subsequent additional tests.

* Apply suggestion to ggml.h

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* Apply suggestion to tests/test-backend-sched.cpp

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* Apply suggestion to tests/test-backend-sched.cpp

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* Fixes node compatibility check to not take buffer locations into account.
This led to skipped tests due to incorrectly rejected ops due to wrong buffer locations in multi-GPU (which is one of the scheduler behaviors under test)

* Apply batched suggestions from gg code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Completes callsite change of gg-suggestion

* Switch test-sleep from std::chrono::steady_clock::now() to ggml_time_us
to avoid mac clock drift and spurious test failures.
This makes the test a bit nonsensical on CPU (same clock checks itself),
but the test still has merit on the other backends where
independent device clocks are used.

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
authored and GitHub committed 2026-11-19 16:40:13 +02:00
1 parent 8af68d486a
commit ac830a9cbb
19 files changed
+1469 -4

No files matched your search

+31
View File
@@ -9657,6 +9657,33 @@ struct test_fill : public test_case {
}
};
// GGML_OP_SLEEP (for scheduler testing only)
struct test_sleep : public test_case {
const ggml_type type;
const std::array<int64_t, 4> ne;
const int32_t us;
std::string vars() override { return VARS_TO_STR3(type, ne, us); }
// the op only delays, the output aliases the input and must match it exactly
double max_nmse_err() override { return 0.0; }
test_sleep(int32_t us = 100, ggml_type type = GGML_TYPE_F32,
std::array<int64_t, 4> ne = { 10, 10, 4, 3 })
: type(type), ne(ne), us(us) {}
ggml_tensor * build_graph(ggml_context * ctx) override {
ggml_tensor * a = ggml_new_tensor_4d(ctx, type, ne[0], ne[1], ne[2], ne[3]);
ggml_set_name(a, "a");
ggml_tensor * out = ggml_sleep(ctx, a, us);
ggml_set_name(out, "out");
return out;
}
};
// GGML_OP_SOLVE_TRI
struct test_solve_tri : public test_case {
const ggml_type type;
@@ -12996,6 +13023,10 @@ static std::vector<std::unique_ptr<test_case>> make_test_cases_eval() {
test_cases.emplace_back(new test_fill(-152.0f, GGML_TYPE_F16, { 800, 600, 4, 4 }));
test_cases.emplace_back(new test_fill(3.5f, GGML_TYPE_F16, { 2048, 512, 2, 2 }));
test_cases.emplace_back(new test_sleep(0));
test_cases.emplace_back(new test_sleep(10, GGML_TYPE_F16, { 128, 4, 2, 2 }));
test_cases.emplace_back(new test_sleep(10, GGML_TYPE_F32, { 128, 4, 2, 2 }));
for (ggml_type type : {GGML_TYPE_F32, GGML_TYPE_F16, GGML_TYPE_BF16}) {
test_cases.emplace_back(new test_diag(type));
test_cases.emplace_back(new test_diag(type, { 79, 1, 19, 13 }));