server : refactored the task processing logic (#5065)

* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
This commit is contained in:
Xuan Son Nguyen authored and GitHub committed 2024-01-26 14:42:20 +02:00
1 parent 413e7b0559
commit 48c857aa10
5 files changed
+876 -692

No files matched your search

+159 -690
View File
File diff suppressed because it is too large. Load diff