Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,333 Commits 5 Branches 0 Tags
35c9b1f39ebe5a7bb83986d64415a079218be78d
Commit Graph
66 Commits
Author SHA1 Message Date
Georgi GerganovandXuan Son Nguyen 2f966b8ed8 clip : use FA (#16837) 
* clip : use FA

* cont : add warning about unsupported ops

* implement "auto" mode for clip flash attn

* clip : print more detailed op support info during warmup

* cont : remove obsolete comment [no ci]

* improve debugging message

* trailing space

* metal : remove stray return

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2025-11-02 21:21:48 +01:00
Ilia IlmerandGeorgi Gerganov 9ad4f1931e metal : add CONV_TRANSPOSE_2D (#16542) 
* initial: headers and metal-device.cpp updates

* adding conv_transpose_2d

* fix type

* fix type: int32->int64

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* add checks for src[0] and src[1]; add type checks

* Update ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* add more tests, add optimization to threading

* add dynamic memory allocation in metal

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-10-17 09:33:58 +03:00
Sam/SamuelandGeorgi Gerganov f4ce81c45e metal: optimise GGML_OP_SUM (#16559) 
* optimise GGML_OP_SUM

* add non-contiguous tests by permuting the input

* change tests to require full contiguity of OP_SUM

* cuda : add check GGML_OP_SUM

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-10-15 17:05:56 +03:00
Georgi Gerganov fa882fd2b1 metal : avoid using Metal's gpuAddress property (#16576) 
* metal : avoid using Metal's gpuAddress property

* metal : fix rope kernels buffer check
 2025-10-14 20:33:05 +03:00
Georgi Gerganov e60f241eac metal : FA support F32 K and V and head size = 32 (#16531) 
* metal : FA support F32 K and V and head size = 32

* graph : remove obsolete comment [no ci]
 2025-10-13 23:07:57 +03:00
Sam/Samuel 3f750f8d76 metal: add support for opt_step_sgd (#16539) 
* metal: add support for opt_step_sgd

* add newline to pass EditorConfig check
 2025-10-13 11:25:02 +03:00
Sam/Samuel a31cf36ad9 metal : add opt_step_adamw and op_sum (#16529) 
* scaffold to support opt step adamw on metal (not written so far)

* add opt-step-adamw kernel for metal

* pass op->src[4] as a separate buffer to the pipeline

* add bounds check to opt-step-adamw kernel

* complete scaffold for GGML_OP_SUM

* naive GGML_OP_SUM kernel

* remove unwanted comment

* change OP_SUM capability gate

* Add has_simdgroup_reduction to both ops to pass CI
 2025-10-12 21:43:14 +03:00
Georgi Gerganov 8ae32dc9ec metal : various optimizations + refactoring (#16446) 
* metal : ssm_scan minor opts

* metal : get_rows optimize

* metal : cpy optimize

* metal : ssm_conv opt

* metal : ssm_scan simplify

* metal : ssm_Scan opt
 2025-10-07 08:21:40 +03:00
Sigbjørn Skjæret adc76347d7 ggml : check cuda and metal argsort limits and add test (#16323) 
* check cuda argsort limits and add test

* add metal check
 2025-09-29 11:09:00 +02:00
Georgi Gerganov 6a2c6145a0 metal : extend mat-mat multiplication support (#16225) 
* metal : support mul_mm with src1->type == GGML_TYPE_F16

* metal : support mul_mm_id with src1->type == GGML_TYPE_F16

[no ci]

* metal : mul_mm support ne00 % 32 != 0

* metal : support mul_mm_id with ne00 % 32 != 0

* cont : remove unnecessary unrolls

* cont : simplify data loading

* metal : optimize mul_mm when output bounds checks are not needed
 2025-09-28 09:34:44 +03:00
Georgi Gerganov 54dbc37053 metal : report OOM errors (#16274) 2025-09-26 14:14:28 +03:00
Georgi Gerganov dfcd53f7ec metal : fuse NORM + MUL + ADD, support non-multiples of 4 (#16220) 
* metal : fuse NORM + MUL + ADD

* metal : support norms of non-multiple of 4

* cont : fix comment [no ci]
 2025-09-25 11:30:16 +03:00
Georgi Gerganov b213fce89b metal : improve F32, F16 and BF16 mat-vec multiplication (#16057) 
* metal : improve F32, F16 and BF16 mat-vec multiplication

ggml-ci

* metal : make the NSG a function constant in mul_mv kernels

ggml-ci
 2025-09-18 12:33:45 +03:00
Jhen-Jie Hong e00f3fd8ff metal : avoid call free for non-owned buffer (#16067) 2025-09-18 10:06:48 +03:00
Georgi Gerganov f2f28380ea metal : handle nil cv during pipeline creation (#16065) 
ggml-ci
 2025-09-18 10:03:24 +03:00
Georgi Gerganov 0320ac5264 metal : refactor + optimize v2 (#15995) 
* metal : improve naming

* metal : refactor device

ggml-ci

* cont : props

ggml-ci

* metal : apply ggml_mem_ranges_t

ggml-ci

* metal : remove GGML_METAL_USE_BF16

ggml-ci

* metal : refactor device buffer

ggml-ci

* cont : fix naming

* metal : sync before destroying the backend

ggml-ci

* metal : refactor context

ggml-ci

* metal : migrate ggml-metal.m to ggml-metal.cpp

ggml-ci

* metal : adjust ops API

ggml-ci

* metal : use C++ to store piplienes

ggml-ci

* metal : migrate ops to separate functions

ggml-ci

* metal : add ggml_metal_library_t

ggml-ci

* metal : improve naming

ggml-ci

* metal : cleanp

ggml-ci

* metal : add support for GGML_OP_LOG

ggml-ci

* metal : fix error handling

ggml-ci
 2025-09-17 20:38:12 +03:00