* ggml-rpc: fail gracefully on unreachable endpoints and failed handshakes rpc_dispatcher::start() aborted the whole process (GGML_ABORT -> SIGABRT) when the endpoint could not be parsed, the transport failed to initialise, the socket could not connect, or the HELLO handshake failed. A typo in --rpc or a temporarily down node killed the host application, and the reason went to a log stream the host may never see. Make the connect-time failure reportable instead of fatal: start() returns bool (LOG_ERROR + false; the dispatcher thread only starts after a successful handshake), negotiate_hello() returns false on a half-open socket, get_dispatcher() returns nullptr and does not cache failed attempts, and every consumer treats nullptr as "endpoint did not answer" with the conservative behaviour of its domain (allocation -> nullptr, alloc-size -> lower bound, device memory -> 0/0, events -> no-op, device count -> 0, and add_server() already maps 0 devices to nullptr). Mid-stream failures keep the existing RPC_STATUS_ASSERT by design (fail fast against silent corruption) and are intentionally out of scope. Field motivation: on Android (llama.rn + phone RPC cluster) an unreachable worker crashed the app; the crash buffer showed abort <- ggml_abort <- rpc_dispatcher::start <- ggml_backend_rpc_add_server. * tests : add a regression test for unreachable rpc endpoints Calls ggml_backend_rpc_add_server with a socket that nothing listens on and with a malformed endpoint, and expects nullptr from both. Without the fix in this branch the first call aborts the process, so the test cannot even fail cleanly. * ggml-rpc remove low-value comments * tests : drop the internal device-count check after the rpc registry refactor --------- Co-authored-by: figkcoci-debug <figkcoci-debug@users.noreply.github.com>
26 lines
1.2 KiB
CMake
26 lines
1.2 KiB
CMake
set(TARGET ggml-rpc-server)
|
|
add_executable(${TARGET} rpc-server.cpp)
|
|
target_link_libraries(${TARGET} PRIVATE ggml)
|
|
target_compile_features(${TARGET} PRIVATE cxx_std_17)
|
|
|
|
if (LLAMA_BUILD_TESTS AND UNIX AND NOT GGML_BACKEND_DL)
|
|
add_executable(test-rpc-multi-server ${PROJECT_SOURCE_DIR}/tests/test-rpc-multi-server.cpp)
|
|
target_link_libraries(test-rpc-multi-server PRIVATE ggml ggml-rpc)
|
|
target_include_directories(test-rpc-multi-server PRIVATE ${PROJECT_SOURCE_DIR}/ggml/src)
|
|
add_test(
|
|
NAME test-rpc-multi-server
|
|
COMMAND bash ${PROJECT_SOURCE_DIR}/tests/test-rpc-multi-server.sh
|
|
$<TARGET_FILE:ggml-rpc-server>
|
|
$<TARGET_FILE:test-rpc-multi-server>)
|
|
set_property(TEST test-rpc-multi-server PROPERTY LABELS main)
|
|
add_executable(test-rpc-unreachable ${PROJECT_SOURCE_DIR}/tests/test-rpc-unreachable.cpp)
|
|
target_link_libraries(test-rpc-unreachable PRIVATE ggml ggml-rpc)
|
|
target_include_directories(test-rpc-unreachable PRIVATE ${PROJECT_SOURCE_DIR}/ggml/src)
|
|
add_test(NAME test-rpc-unreachable COMMAND test-rpc-unreachable)
|
|
set_property(TEST test-rpc-unreachable PROPERTY LABELS main)
|
|
endif()
|
|
|
|
if(LLAMA_TOOLS_INSTALL)
|
|
install(TARGETS ${TARGET} RUNTIME)
|
|
endif()
|