llama.cpp

2,985 Commits 4 Branches 0 Tags

2 Commits

Author	SHA1	Message	Date
fairydreamingandStanisław Szymczyk	9b82476ee9	Add missing inference support for GPTNeoXForCausalLM (Pythia and GPT-NeoX base models) (#7461 ) * convert-hf : add conversion of bloom-style qkv tensor to gpt-style qkv (code borrowed from BloomModel) * llama : add inference support for LLM_ARCH_GPTNEOX * llama : add model types for every Pythia variant and GPT-NeoX Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>	2024-05-23 11:49:53 +02:00
fairydreamingandStanisław Szymczyk	27b040691c	llama : use n_embd_head_v when reshaping kqv (#7327 ) * llama : use n_embd_head_v instead of n_embd_head_k when reshaping kqv * llama : use n_embd_v_gqa and n_embd_head_v instead of n_embd_k_gqa and n_embd_head_k when making a view of cached value vectors. --------- Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>	2024-05-17 14:24:38 +03:00

Author

SHA1

Message

Date

fairydreamingandStanisław Szymczyk

9b82476ee9

Add missing inference support for GPTNeoXForCausalLM (Pythia and GPT-NeoX base models) (#7461 )

* convert-hf : add conversion of bloom-style qkv tensor to gpt-style qkv (code borrowed from BloomModel)

* llama : add inference support for LLM_ARCH_GPTNEOX

* llama : add model types for every Pythia variant and GPT-NeoX

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>

2024-05-23 11:49:53 +02:00

fairydreamingandStanisław Szymczyk

27b040691c

llama : use n_embd_head_v when reshaping kqv (#7327 )

* llama : use n_embd_head_v instead of n_embd_head_k when reshaping kqv

* llama : use n_embd_v_gqa and n_embd_head_v instead of n_embd_k_gqa and n_embd_head_k when making a view of cached value vectors.

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>

2024-05-17 14:24:38 +03:00