Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,079 Commits 5 Branches 0 Tags
416e7c7f477c06b3ad208067ace057c8d8be745f
Commit Graph
18 Commits
Author SHA1 Message Date
Georgi Gerganov 45c6ef7307 metal : support argsort for ne00 > 1024 (#17247) 
* metal : refactor argsort

* cont : sort chunks

* cont : merge sorted buckets

* cont : cleanup
 2025-11-14 09:36:06 +02:00
bagheerabghiraGeorgi Gerganov
 		0cfb19166b metal: accelerated conv2d (#17175) 
* metal: accelerated conv2d

* cont : cleanup

---------

Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-11-13 13:32:44 +02:00
Ruben OrtlamandGeorgi Gerganov d2a2673dd1 vulkan: fix shmem overrun in mmq id shader (#16873) 
* vulkan: fix shmem overrun in mmq id shader

* metal : fix mul_mm_id

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-10-31 08:14:49 +01:00
JJJYmmmThireus ☠yairpatchLETS-BEESigbjørn SkjæretXuan Son NguyenGeorgi Gerganov
 		d261223d24 model: add support for qwen3vl series (#16780) 
* support qwen3vl series.

Co-authored-by: Thireus ☠ <Thireus@users.noreply.github.com>
Co-authored-by: yairpatch <yairpatch@users.noreply.github.com>
Co-authored-by: LETS-BEE <LETS-BEE@users.noreply.github.com>

* bugfix: fix the arch check for qwen3vl-moe.

* use build_ffn

* optimize deepstack structure

* optimize deepstack feature saving

* Revert "optimize deepstack feature saving" for temporal fix

This reverts commit f321b9fdf13e59527408152e73b1071e19a87e71.

* code clean

* use fused qkv in clip

* clean up / rm is_deepstack_layers for simplification

* add test model

* move test model to "big" section

* fix imrope check

* remove trailing whitespace

* fix rope fail

* metal : add imrope support

* add imrope support for sycl

* vulkan: add imrope w/o check

* fix vulkan

* webgpu: add imrope w/o check

* Update gguf-py/gguf/tensor_mapping.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix tensor mapping

---------

Co-authored-by: Thireus ☠ <Thireus@users.noreply.github.com>
Co-authored-by: yairpatch <yairpatch@users.noreply.github.com>
Co-authored-by: LETS-BEE <LETS-BEE@users.noreply.github.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-10-30 16:19:14 +01:00
Ilia IlmerandGeorgi Gerganov 9ad4f1931e metal : add CONV_TRANSPOSE_2D (#16542) 
* initial: headers and metal-device.cpp updates

* adding conv_transpose_2d

* fix type

* fix type: int32->int64

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update ggml/src/ggml-metal/ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* add checks for src[0] and src[1]; add type checks

* Update ggml-metal.metal

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* add more tests, add optimization to threading

* add dynamic memory allocation in metal

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-10-17 09:33:58 +03:00
Sam/Samuel 3f750f8d76 metal: add support for opt_step_sgd (#16539) 
* metal: add support for opt_step_sgd

* add newline to pass EditorConfig check
 2025-10-13 11:25:02 +03:00
Sam/Samuel a31cf36ad9 metal : add opt_step_adamw and op_sum (#16529) 
* scaffold to support opt step adamw on metal (not written so far)

* add opt-step-adamw kernel for metal

* pass op->src[4] as a separate buffer to the pipeline

* add bounds check to opt-step-adamw kernel

* complete scaffold for GGML_OP_SUM

* naive GGML_OP_SUM kernel

* remove unwanted comment

* change OP_SUM capability gate

* Add has_simdgroup_reduction to both ops to pass CI
 2025-10-12 21:43:14 +03:00
Georgi Gerganov b2c08c9ec4 metal : mark FA blocks (#16372) 
* metal : better unroll in the FA kernels

* metal : index FA blocks

* tests : restore [no ci]

* metal : prevent division by zero in FA kernels

* metal : fix -INF detection logic
 2025-10-08 10:57:53 +03:00
Georgi Gerganov 0a319bb75e metal : add support for non-padded FA KV (#16148) 
* metal : pad K, V and Mask when needed

* cont : simplify

* cuda : add TODO about KV padding requirement

* metal : add comments

* metal : remove mask padding requirement
 2025-10-07 08:23:30 +03:00
Georgi Gerganov 8ae32dc9ec metal : various optimizations + refactoring (#16446) 
* metal : ssm_scan minor opts

* metal : get_rows optimize

* metal : cpy optimize

* metal : ssm_conv opt

* metal : ssm_scan simplify

* metal : ssm_Scan opt
 2025-10-07 08:21:40 +03:00
Georgi Gerganov 35fb82497e metal : dynamic simdgroups for MV kernels (#16340) 
* metal : dynamic simdgroups for MV kernels

* cont : minor
 2025-09-30 11:03:23 +03:00
Georgi Gerganov 6a2c6145a0 metal : extend mat-mat multiplication support (#16225) 
* metal : support mul_mm with src1->type == GGML_TYPE_F16

* metal : support mul_mm_id with src1->type == GGML_TYPE_F16

[no ci]

* metal : mul_mm support ne00 % 32 != 0

* metal : support mul_mm_id with ne00 % 32 != 0

* cont : remove unnecessary unrolls

* cont : simplify data loading

* metal : optimize mul_mm when output bounds checks are not needed
 2025-09-28 09:34:44 +03:00
Georgi Gerganov dfcd53f7ec metal : fuse NORM + MUL + ADD, support non-multiples of 4 (#16220) 
* metal : fuse NORM + MUL + ADD

* metal : support norms of non-multiple of 4

* cont : fix comment [no ci]
 2025-09-25 11:30:16 +03:00
Georgi Gerganov 02a6a82ae7 metal : restore im2col perf (#16219) 2025-09-25 11:29:08 +03:00
Sigbjørn SkjæretGeorgi GerganovJeff Bolz
 		3ecb2f671a ggml : implement set_rows with i32 index (#16159) 
* implement set_rows with i32 index

* template fix

* test quantized path

warnings--

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* forgotten name change

* deduplicate cuda/sycl and test-fix

* indent++

* vulkan: support set_rows with i32 index type (#16162)

* disable i32 index for webgpu for now

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-09-22 19:13:00 +02:00
Georgi Gerganov 703f9e32c4 metal : use function constants for mul_mv_ext kernels (#16074) 
* metal : use function constants for mul_mv_ext kernels

ggml-ci

* metal : remove NW template argument

ggml-ci

* metal : adjust constants

ggml-ci
 2025-09-18 16:28:41 +03:00
Georgi Gerganov b213fce89b metal : improve F32, F16 and BF16 mat-vec multiplication (#16057) 
* metal : improve F32, F16 and BF16 mat-vec multiplication

ggml-ci

* metal : make the NSG a function constant in mul_mv kernels

ggml-ci
 2025-09-18 12:33:45 +03:00
Georgi Gerganov 0320ac5264 metal : refactor + optimize v2 (#15995) 
* metal : improve naming

* metal : refactor device

ggml-ci

* cont : props

ggml-ci

* metal : apply ggml_mem_ranges_t

ggml-ci

* metal : remove GGML_METAL_USE_BF16

ggml-ci

* metal : refactor device buffer

ggml-ci

* cont : fix naming

* metal : sync before destroying the backend

ggml-ci

* metal : refactor context

ggml-ci

* metal : migrate ggml-metal.m to ggml-metal.cpp

ggml-ci

* metal : adjust ops API

ggml-ci

* metal : use C++ to store piplienes

ggml-ci

* metal : migrate ops to separate functions

ggml-ci

* metal : add ggml_metal_library_t

ggml-ci

* metal : improve naming

ggml-ci

* metal : cleanp

ggml-ci

* metal : add support for GGML_OP_LOG

ggml-ci

* metal : fix error handling

ggml-ci
 2025-09-17 20:38:12 +03:00