llama.cpp

2,644 Commits 4 Branches 0 Tags

2 Commits

Author	SHA1	Message	Date
Pedro CuencaandJared Van Bortel	b75c38166c	convert : allow conversion of Mistral HF models (#6144 ) * Allow conversion of Mistral HF models * Homogenize Llama, Mistral, Mixtral under the same entry. * Fix tokenizer, permute tensors * Use sentencepiece tokenizer, or fall back to hfft. * convert-hf : small fix for mypy * convert-hf : fix duplicated block_count * convert-hf : add vocab size to metadata --------- Co-authored-by: Jared Van Bortel <jared@nomic.ai>	2024-03-29 09:15:00 +02:00
Pedro Cuenca	e097633f63	convert-hf : fix exception in sentencepiece with added tokens (#6320 )	2024-03-26 14:32:19 +02:00

Author

SHA1

Message

Date

Pedro CuencaandJared Van Bortel

b75c38166c

convert : allow conversion of Mistral HF models (#6144 )

* Allow conversion of Mistral HF models

* Homogenize Llama, Mistral, Mixtral under the same entry.

* Fix tokenizer, permute tensors

* Use sentencepiece tokenizer, or fall back to hfft.

* convert-hf : small fix for mypy

* convert-hf : fix duplicated block_count

* convert-hf : add vocab size to metadata

---------

Co-authored-by: Jared Van Bortel <jared@nomic.ai>

2024-03-29 09:15:00 +02:00

Pedro Cuenca

e097633f63

convert-hf : fix exception in sentencepiece with added tokens (#6320 )

2024-03-26 14:32:19 +02:00