llama.cpp/ggml/src at 75f460ac289e61eb3c2bb63c9487794a1ed514d1 - llama.cpp - Gitea: Git with a cup of tea

Lumpiasty/llama.cpp

Files

T

History

Adrien Gallouët 37a77fb057 ggml : optimize AMX (#24806 )

Flatten the partition over n_batch * M so every thread participates in
the quantization

    | CPU                             | Model                         | Test   |   t/s OLD |   t/s NEW |   Speedup |
    |:--------------------------------|:------------------------------|:-------|----------:|----------:|----------:|
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw  | pp512  |    730.71 |    779.86 |      1.07 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw  | tg128  |     87.88 |     86.79 |      0.99 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | pp512  |    725.09 |   1023.31 |      1.41 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | tg128  |     83.64 |     83.62 |      1.00 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0              | pp512  |    820.51 |    924.05 |      1.13 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0              | tg128  |     90.59 |     92.46 |      1.02 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1              | pp512  |    776.88 |    872.79 |      1.12 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1              | tg128  |     89.39 |     90.94 |      1.02 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M            | pp512  |    719.28 |   1009.27 |      1.40 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M            | tg128  |     80.62 |     80.86 |      1.00 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S            | pp512  |    732.29 |   1077.29 |      1.47 |
    | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S            | tg128  |     86.42 |     83.53 |      0.97 |

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

2026-06-20 13:43:06 +03:00

..

vulkan: add get/set tensor 2d functions (#22514 )

2026-04-30 17:37:13 +02:00

vulkan: add get/set tensor 2d functions (#22514 )

2026-04-30 17:37:13 +02:00

ggml : optimize AMX (#24806 )

2026-06-20 13:43:06 +03:00

Ggml/cuda col2im 1d (#24417 )

2026-06-18 22:23:01 +02:00

hexagon: support for op-trace (fine-grain tracing of HVX/HMX/DMA events) (#24592 )

2026-06-18 08:35:02 -07:00

ggml: backend-agnostic tensor parallelism (experimental) (#19378 )

2026-04-09 16:42:19 +02:00

metal : check for BF16 support in concat kernel (#24747 )

2026-06-18 09:16:06 +03:00

ggml-cuda: native bf16 flash attention for vec kernel (#20525 )

2026-03-22 11:05:51 +01:00

opencl: optimize mul_mat_f16_f32_l4 for decode (#24504 )

2026-06-16 23:21:26 -07:00

openvino: OV 2026.2, context-shift, Q5_1 support, gemma4 dense/embedding, and -fa off (#24503 )

2026-06-17 09:11:21 +03:00

rpc : keep last_graph_uid in the device context (#23273 )

2026-05-19 09:42:36 +03:00

[SYCL] rename GGML_SYCL_SUPPORT_LEVEL_ZERO (#24719 )

2026-06-18 11:18:26 +03:00

ggml-virtgpu : include missing mutex header (#22810 )

2026-05-10 17:32:41 +02:00

vulkan: record actual memory properties during buffer creation (#24326 )

2026-06-17 11:14:48 +02:00

ggml-webgpu: add adapter toggles for F16 on Vulkan + NVIDIA

2026-06-20 08:12:32 +09:00

vulkan: add get/set tensor 2d functions (#22514 )

2026-04-30 17:37:13 +02:00

ggml-zendnn : fixed naming of matmul function (#20964 )

2026-05-27 00:59:35 +02:00

CMakeLists.txt

ggml-cpu: Conditionally enable power11 backend based on compiler support (#24687 )

2026-06-18 02:45:19 +08:00

ggml-alloc.c

ggml-alloc: fix out-of-bounds read in ggml_dyn_tallocr_remove_block (ggml/1492)

2026-05-25 12:38:01 +03:00

ggml-backend-dl.cpp

hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150 )

2026-01-29 12:33:21 -08:00

ggml-backend-dl.h

hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150 )

2026-01-29 12:33:21 -08:00

ggml-backend-impl.h

ggml: backend-agnostic tensor parallelism (experimental) (#19378 )

2026-04-09 16:42:19 +02:00

ggml-backend-meta.cpp

Remove padding and multiple D2D copies for MTP (#24086 )

2026-06-10 23:21:16 +05:30

ggml-backend-reg.cpp

ggml : skip already registered backends and devices (#22296 )

2026-04-28 10:02:32 +03:00

ggml-backend.cpp

ggml : Check the right iface method before using the fallback 2d get (#23514 )

2026-05-23 12:49:24 +02:00

ggml-common.h

ggml: add Q1_0 1-bit quantization support (CPU) (#21273 )

2026-04-06 20:55:21 +02:00

ggml-impl.h

ggml: add graph_reused (#21764 )

2026-04-16 17:21:28 +08:00

ggml-opt.cpp

fix: free ctx_copy in ggml_opt_free to plug per-training-session leak (#21592 )

2026-04-08 17:40:15 +02:00

ggml-quants.c

ggml : Parallelize quant LUT init (#23595 )

2026-05-25 10:15:46 +03:00

ggml-quants.h

ggml: add Q1_0 1-bit quantization support (CPU) (#21273 )

2026-04-06 20:55:21 +02:00

ggml-threading.cpp

ggml : build backends as libraries (#10256 )

2024-11-14 18:04:35 +01:00

ggml-threading.h

remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797 )

2024-12-12 19:02:49 +01:00

ggml.c

mtmd, arg: fix utf8 handling on windows (#24779 )

2026-06-19 22:28:38 +02:00

ggml.cpp

ggml : Print backtrace on uncaught C++ exceptions (ggml/1232)

2025-06-01 13:43:57 +03:00

gguf.cpp

ggml: gguf_init_from_callback and gguf_init_from_buffer (#22341 )

2026-05-25 11:33:29 +02:00