Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,575 Commits 5 Branches 0 Tags
0bd1212a43561a55d89d468b7ca5265647dab1fd
Commit Graph
32 Commits
Author SHA1 Message Date
Johannes Gäßler 0bd1212a43 CUDA: fix replacment of bad archs in CMake (#18457) 2025-12-29 17:58:20 +01:00
Johannes Gäßler e70e640db3 CUDA: Blackwell features for non-native builds (#18436) 2025-12-29 09:35:42 +01:00
Aman Gupta 07a0c4ba92 Revert "ggml-cuda: use CMAKE_CUDA_ARCHITECTURES if set when GGML_NATIVE=ON (#18413)" (#18426) 2025-12-28 20:53:36 +08:00
QDelta 4fd59e8427 ggml-cuda: use CMAKE_CUDA_ARCHITECTURES if set when GGML_NATIVE=ON (#18413) 2025-12-28 09:33:14 +08:00
Aman Gupta 06705fdcb3 ggml-cuda: Use same regex for GGML_NATIVE=OFF (#18407) 2025-12-27 19:56:27 +08:00
Aman Gupta 85c40c9b02 ggml-cuda: fix regex for arch list (#18371) 
* ggml-cuda: fix regex for arch list

* make regex exact
 2025-12-26 01:35:14 +08:00
Aman GuptaandAman Gupta <aman> b0fb0f0aee ggml-cuda: fix blackwell native builds (#18361) 
* ggml-cuda: fix blackwell native builds

Replace 12x in native architectures by 12xa

* replace for GGML_NATIVE=OFF too

* only replace for native

* remove 120f-virtual for default compilation

---------

Co-authored-by: Aman Gupta <aman>
 2025-12-25 22:12:11 +08:00
Aman GuptaandAman Gupta <aman> c8a2417d7b CUDA: experimental native mxfp4 support for blackwell (#17906) 
* CUDA: experimental native mxfp4 support for blackwell

* optimize load_tiles

* optimize quantize_mxfp4

* cleanup

* first pass review: formatting

* use interleaved layout for mma

* mmq: add assert for size

* use __nv_fp4x4_e2m1

* use iter_k as 512, cleanup

* Use 1200 as blackwell instead of 1000

* address review comments

* mmq: fix stride

* quantize.cu: use reference impl of e8m0 scale

* address review comments

* add 120f-virtual + minor fixes

---------

Co-authored-by: Aman Gupta <aman>
 2025-12-24 22:28:26 +08:00
Aman Gupta c1b187688d CUDA: skip fusion for repeating adds in bias (#17080) 2025-11-08 16:58:05 +08:00
Johannes Gäßler 11f0af5504 CUDA: faster tile FA, add oob checks, more HSs (#16492) 2025-10-11 20:54:32 +02:00
Gregor Jasny fa6383ca7e CUDA : conditionally add cuda architectures (ggml/1341) 2025-09-20 13:02:14 +03:00
Aman Gupta a972faebed CUDA: Add mul_mat_id support for the mmf kernel (#15767) 
* CUDA: Add mul_mat_id support the mmf

Add support for mul_mat_id for bs < 16

* Review: use warp_size, fix should_use_mmf condition

* Launch one block per expert, stride along n_expert_used

* templatize mul_mat_id

* Pad shmem to 16 bytes, add helper function mul_mat_f_switch_ids

* Reduce compile times by dividing mmf into f16, bf16 and f32 variants

* Divide mmf by ncols_dst

* Add missing files

* Fix MUSA/HIP builds
 2025-09-09 14:38:02 +08:00
matiaslin 5a0e3ef6f0 cuda: Add cublasLt_static linking when GGML_STATIC is enabled (#15622) 
Prior to this change, we faced undefined cublasLt references when
attempting to compile 'llama-cli' with GGML_STATIC=ON on Linux.

We add linking with CUDA::cublasLt_static when CUDA version is greater
than 10.1.
 2025-08-28 02:32:36 +02:00
Johannes Gäßler 7a6e91ad26 CUDA: replace GGML_CUDA_F16 with CUDA arch checks (#15433) 2025-08-20 16:58:49 +02:00
Aman Gupta efe3a90996 CUDA cmake: add -lineinfo for easier debug (#15260) 2025-08-12 17:21:45 +08:00
R0CKSTAR 48b86c4fdb cuda: remove linking to cublasLt (#14790) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2025-07-22 07:45:26 +08:00
Johannes Gäßler 0cf6725e9f CUDA: FA support for Deepseek (Ampere or newer) (#13306) 
* CUDA: FA support for Deepseek (Ampere or newer)

* do loop unrolling via C++ template
 2025-05-09 13:34:58 +02:00
Johannes Gäßler 141a908a59 CUDA: mix virt/real CUDA archs for GGML_NATIVE=OFF (#13135) 2025-05-06 23:35:51 +02:00
Diego Devesa d7a14c42a1 build : fix build info on windows (#13239) 
* build : fix build info on windows

* fix cuda host compiler msg
 2025-05-01 21:48:08 +02:00
Erik Scholz 80c41ddd8f CUDA: compress mode option and default to size (#12029) 
cuda 12.8 added the option to specify stronger compression for binaries, so we now default to "size".
 2025-03-01 12:57:22 +01:00
Johannes Gäßler a28e0d5eb1 CUDA: app option to compile without FlashAttention (#12025) 2025-02-22 20:44:34 +01:00
PureJourneyandJohannes Gäßler ecc8e3aeff CUDA: correct the lowest Maxwell supported by CUDA 12 (#11984) 
* CUDA: correct the lowest Maxwell supported by CUDA 12

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2025-02-21 12:21:05 +01:00
Diego Devesa 94b87f87b5 cuda : add ampere to the list of default architectures (#11870) 2025-02-14 15:33:52 +01:00
Johannes GäßlerandDiego Devesa 864a0b67a6 CUDA: use mma PTX instructions for FlashAttention (#11583) 
* CUDA: use mma PTX instructions for FlashAttention

* __shfl_sync workaround for movmatrix

* add __shfl_sync to HIP

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2025-02-02 19:31:09 +01:00
Georgi Gerganov ab96610b1e cmake : enable warnings in llama (#10474) 
* cmake : enable warnings in llama

ggml-ci

* cmake : add llama_get_flags and respect LLAMA_FATAL_WARNINGS

* cmake : get_flags -> ggml_get_flags

* speculative-simple : fix warnings

* cmake : reuse ggml_get_flags

ggml-ci

* speculative-simple : fix compile warning

ggml-ci
 2024-11-26 14:18:08 +02:00
Diego DevesaandGeorgi Gerganov 5931c1f233 ggml : add support for dynamic loading of backends (#10469) 
* ggml : add support for dynamic loading of backends

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-25 15:13:39 +01:00
Diego Devesa 3ee6382d48 cuda : fix CUDA_FLAGS not being applied (#10403) 2024-11-19 14:29:38 +01:00
Diego Devesa d3481e6316 cuda : only use native when supported by cmake (#10389) 2024-11-18 18:43:40 +01:00
Johannes Gäßler ce2e59ba10 CMake: fix typo in comment [no ci] (#10360) 2024-11-17 12:59:38 +01:00
Johannes Gäßler c3ea58aca4 CUDA: remove DMMV, consolidate F16 mult mat vec (#10318) 2024-11-17 09:09:55 +01:00
Johannes Gäßler 467576b6cc CMake: default to -arch=native for CUDA build (#10320) 2024-11-17 09:06:34 +01:00
Diego DevesaGeorgi GerganovR0CKSTAR
 		ae8de6d50a ggml : build backends as libraries (#10256) 
* ggml : build backends as libraries

---------

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: R0CKSTAR <xiaodong.ye@mthreads.com>
 2024-11-14 18:04:35 +01:00