Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,999 Commits 4 Branches 0 Tags
47c786924ad1ab7e91da2cdc72fcdb563780c2bd
Commit Graph
20 Commits
Author SHA1 Message Date
Charles Xu 47c786924a kleidiai : add SME2 f32 kernel (#24414) 
* kleidiai : add SME2 f32 kernel

* enable dynamic scheduling for SME2 f32 kernel
 2026-07-14 13:12:18 +03:00
Charles Xu 3ecfb150a4 kleidiai : dynamic chunck-based scheduling for hybrid execution (#23819) 2026-06-05 10:11:47 +03:00
Justin Bradford 627670601a kleidiai : fix MUL_MAT support for batched (3D) inputs (#20620) 
* kleidiai : fix MUL_MAT support for batched (3D) inputs

The supports_op() check incorrectly rejected MUL_MAT operations with 3D
inputs (ne[2] > 1), but the actual compute_forward_qx() implementation
handles batched inputs correctly via a loop over ne12.

This caused models with Q4_0/Q8_0 weights to crash during graph scheduling
when n_seq_max > 1, because weights were placed in KLEIDIAI buffers during
loading (tested with 2D inputs) but the runtime used 3D inputs.

Also relax the buffer check to allow supports_op() to be called during
weight loading when src[0]->buffer is NULL.

Fixes #20608

* Kleidiai support_ops should only return true for 3D inputs, not also 4D
 2026-03-17 14:03:54 +02:00
Martin KlacerandGeorgi Gerganov cf21cdf36c kleidiai: add data type check to get_tensor_traits (#20639) 
* kleidiai: add data type check to get_tensor_traits

 * Added check for F16 data type into get_tensor_traits path with input data
   not in ggml_backend_cpu_kleidiai_buffer_type format (unsupported for Q4/8)

Signed-off-by: Martin Klacer <martin.klacer@arm.com>
Change-Id: I9aca4b9b8d669d35db6f1dbcc4e080b1919b1de7

* updated ggml/src/ggml-cpu/kleidiai/kleidiai.cpp

updated kleidiai.cpp file as per suggestion

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Signed-off-by: Martin Klacer <martin.klacer@arm.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-16 21:25:54 +02:00
Charles Xu 0cd4f4720b kleidiai : support for concurrent sme and neon kernel execution (#20070) 2026-03-10 09:25:25 +02:00
Charles Xu 2d6c00a9b8 kleidiai: add and integrate SVE 256-bit vector-length kernel (#18458) 
* kleidiai: add and integrate SVE 256-bit vector-length kernel

* updated for review comments
 2025-12-30 14:04:53 +02:00
Charles Xu 8c583242ad kleidiai: add optimized per-channel kernels for Q8_0 (#16993) 2025-11-11 13:20:31 +02:00
Charles Xu d80d6d2400 kleidiai: kernel interface refactoring (#16460) 2025-10-09 10:29:17 +03:00
Charles Xu f1eb1cb1eb kleidiai : fix work size and threads sync for fp16 (#16246) 2025-09-30 10:07:20 +03:00
Charles Xu 2b3efea9a4 kleidiai: fix GGML_ASSERT(*cur_backend_id != -1) failed (#15614) 
* kleidiai: fix GGML_ASSERT(*cur_backend_id != -1) failed

* removes the Whisper-specific check for GET_ROWS support
 2025-09-11 12:45:40 +02:00
Charles Xu c4df49a42d kleidiai: generalize compute_forward_kv_cache to compute_forward_fp16 (#15817) 2025-09-06 22:08:43 +08:00
Charles Xu 4d74393bcc ggml: update kleidiai to v1.13.0 (#15663) 2025-08-31 00:03:42 +08:00
Charles Xu 002cb1bb33 kleidiai: fix unsigned overflow bug (#15150) 
* kleidiai: fix unsigned overflow bug

* address review comments
 2025-08-11 09:59:26 +02:00
Charles Xu 922042601b kleidiai: add support for get_rows (#14676) 
* kleidiai: add support for get_rows

* apply fixes based on code review

* apply more fixes based on code review
 2025-07-21 16:49:52 +03:00
xctanandGeorgi Gerganov f470bc36be ggml-cpu : split arch-specific implementations (#13892) 
* move ggml-cpu-aarch64 to repack

* split quantize_row_q8_0/1

* split helper functions

* split ggml_vec_dot_q4_0_q8_0

* split ggml_vec_dot_q4_1_q8_1

* split ggml_vec_dot_q5_0_q8_0

* split ggml_vec_dot_q5_1_q8_1

* split ggml_vec_dot_q8_0_q8_0

* split ggml_vec_dot_tq1_0_q8_K

* split ggml_vec_dot_tq2_0_q8_K

* split ggml_vec_dot_q2_K_q8_K

* split ggml_vec_dot_q3_K_q8_K

* split ggml_vec_dot_q4_K_q8_K

* split ggml_vec_dot_q5_K_q8_K

* split ggml_vec_dot_q6_K_q8_K

* split ggml_vec_dot_iq2_xxs_q8_K

* split ggml_vec_dot_iq2_xs_q8_K

* split ggml_vec_dot_iq2_s_q8_K

* split ggml_vec_dot_iq3_xxs_q8_K

* split ggml_vec_dot_iq3_s_q8_K

* split ggml_vec_dot_iq1_s_q8_K

* split ggml_vec_dot_iq1_m_q8_K

* split ggml_vec_dot_iq4_nl_q8_0

* split ggml_vec_dot_iq4_xs_q8_K

* fix typos

* fix missing prototypes

* rename ggml-cpu-quants.c

* rename ggml-cpu-traits

* rename arm folder

* move cpu-feats-x86.cpp

* rename ggml-cpu-hbm

* update arm detection macro in quants.c

* move iq quant tables

* split ggml_quantize_mat_q8_0/K

* split ggml_gemv_*

* split ggml_gemm_*

* rename namespace aarch64 to repack

* use weak aliases to replace test macros

* rename GGML_CPU_AARCH64 to GGML_CPU_REPACK

* rename more aarch64 to repack

* clean up rebase leftover

* fix compilation errors

* remove trailing spaces

* try to fix clang compilation errors

* try to fix clang compilation errors again

* try to fix clang compilation errors, 3rd attempt

* try to fix clang compilation errors, 4th attempt

* try to fix clang compilation errors, 5th attempt

* try to fix clang compilation errors, 6th attempt

* try to fix clang compilation errors, 7th attempt

* try to fix clang compilation errors, 8th attempt

* try to fix clang compilation errors, 9th attempt

* more cleanup

* fix compilation errors

* fix apple targets

* fix a typo in arm version of ggml_vec_dot_q4_K_q8_K

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-06-09 16:47:13 +02:00
Dan Johansson 4f711afed5 ggml-cpu: Update KleidiAI to v1.6 and fix include directives (#13509) 
Signed-off-by: Dan Johansson <dan.johansson@arm.com>
 2025-05-13 18:02:28 +03:00
Dan JohanssonandCharles Xu a71a4075cd ggml-cpu: Integrate fp32=bf16xbf16 SME KleidiAI kernel (#13053) 
* ggml-cpu: Integrate fp32=bf16xbf16 SME KleidiAI kernel

Signed-off-by: Dan Johansson <dan.johansson@arm.com>

* * code review fixes

Signed-off-by: Dan Johansson <dan.johansson@arm.com>

* * adds a comment that clarifies barrier usage

Signed-off-by: Dan Johansson <dan.johansson@arm.com>

---------

Signed-off-by: Dan Johansson <dan.johansson@arm.com>
Co-authored-by: Charles Xu <charles.xu@arm.com>
 2025-05-12 13:06:19 +02:00
Dan Johansson 053b3f9aae ggml-cpu : update KleidiAI to v1.5.0 (#12568) 
ggml-cpu : bug fix related to KleidiAI LHS packing

Signed-off-by: Dan Johansson <dan.johansson@arm.com>
 2025-03-25 13:10:18 +02:00
ag2s20150909 9660ffef58 ggml : fix kleidiai build (#12159) 
The libggml API has changed, but this has not been updated.
 2025-03-03 13:54:08 +01:00
Charles Xu c5d91a7400 ggml-cpu: Add CPU backend support for KleidiAI library (#11390) 
* ggml-cpu: Add CPU backend support for KleidiAI library

* Add environmental variable GGML_KLEIDIAI_SME

* Add support for multithread LHS conversion

* Switch kernel selection order to dotprod and i8mm

* updates for review comments

* More updates for review comments

* Reorganize and rename KleidiAI files

* Move ggml-cpu-traits.h to source file

* Update cmake for SME build and add alignment for SME

* Remove append GGML_USE_CPU_KLEIDIAI to the GGML_CDEF_PUBLIC list
 2025-02-20 15:06:51 +02:00