Aman GuptaandJohannes Gäßler
f8f0a47a55
* cuda: reserve space for quantize kv-cache at startup * address review comments * remove forward decl Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * remove assert in ggml-cuda.cu Co-authored-by: Johannes Gäßler <johannesg@5d6.de> --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
8 lines
271 B
Plaintext
8 lines
271 B
Plaintext
#include "common.cuh"
|
|
|
|
void ggml_cuda_flash_attn_ext(ggml_backend_cuda_context & ctx, ggml_tensor * dst);
|
|
|
|
bool ggml_cuda_flash_attn_ext_supported(int device, const ggml_tensor * dst);
|
|
|
|
size_t ggml_cuda_flash_attn_ext_get_alloc_size(int device, const ggml_tensor * dst);
|