Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

spec: Add benchmark-only synthetic speculative acceptance options (#27711)

Browse Source 
* Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli

* Address review comments

* Address review comments

* Add some comments in the code
This commit is contained in:
Gaurav Garg
2026-08-27 13:53:42 +03:00
committed by GitHub
parent deae5ee133
commit 2bb9bddafa
11 changed files with 427 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/speculative.h
+9
View File
@@ -26,6 +26,15 @@ std::string common_speculative_type_to_str(enum common_speculative_type type);
// return the max number of draft tokens based on the speculative parameters
int32_t common_speculative_n_max(const common_params_speculative * spec);
// return the max number of draft tokens from the initialized implementations
int32_t common_speculative_n_max(const common_speculative * spec);
// validate and resolve the unconditional synthetic acceptance rates
std::vector<double> common_speculative_synth_rates_resolve(const common_params_speculative * spec, int32_t n_max);
// return the conditional synthetic acceptance probabilities
const std::vector<double> & common_speculative_get_synth_probs(const common_speculative * spec);
common_params common_base_params_to_speculative(const common_params & params);
struct common_speculative_output_limits {