server: (router) move model downloading to dedicated process (#24834)

* server: real-time model load progress tracking via /models/sse

* update docs

* server: move model download to child process

* rm unused

* fix most problems

* clean up

* nit fixes

* fix test case

* do not detact() thread

* shorter MODEL_DOWNLOAD_TIMEOUT in test

* throttle
This commit is contained in:
Xuan-Son Nguyen
2026-06-22 18:24:04 +02:00
committed by GitHub
parent 6ee0f65793
commit 721354fbdf
9 changed files with 312 additions and 152 deletions
+6
View File
@@ -931,6 +931,8 @@ private:
bool sleeping = false;
int64_t t_last_load_progress_ms = 0;
void destroy() {
spec.reset();
ctx_dft.reset();
@@ -1244,6 +1246,10 @@ private:
}
if (has_mmproj) {
if (callback_state) {
callback_state(SERVER_STATE_LOADING, {{"stage", "mmproj_model"}});
}
if (!is_resume) {
mtmd_helper_log_set(common_log_default_callback, nullptr);
}