* Enable backend sampling with token speculation
* Clamp the mask sum before converting it into the sampled index
* Add a numeric context parameter declaring the maximum outputs one sequence
* More fixes
* Don't reuse memory for output views.
* Match dist between CPU and GPU
* Fix CPU and backend sampling mismatches
* Simpify some of the changes
* Fix tests on Vulkan
* More test fixes
* Rebase changes
* Rebase and address review comments
* Address review comments
* Address review comments
* Update src/llama-sampler.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Resolve -1 to 1024 instead of ctx-len for samplers
Because of backend-sampling we initialize samplers before the complete
llama_context is there. Therefore, we cannot infer the resolved context
length yet at the time we construct the samplers.
* Shared default of 64 for history-based samplers, remove context_size
