Multi GPU support, CUDA refactor, CUDA scratch buffer (#1703)

Browse Source

* CUDA multi GPU + scratch

ggml_cuda_compute_forward

Tensor parallelism

ggml_cuda_add

ggml_cuda_rms_norm

ggml_cuda_silu

CUDA scratch buffer

--main-gpu CLI option

...

This commit is contained in:

Johannes Gäßler

2023-06-06 21:33:23 +02:00

committed by GitHub

parent 44f906e853

commit 17366df842

12 changed files with 1221 additions and 544 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

ggml-cuda.cu

+889 -430

File diff suppressed because it is too large Load Diff