hexagon: add support for CONCAT op (#23648)

* hexagon: add support for CONCAT with optimized concat_2d_transposed

qwen3.5 models are quite heavy on the CONCAT with large and transposed src1.

* hex-concat: use fastdiv in generic version

* hex-concat: make checks for transposed a bit more readable

* hex-concat: reoder dma ops for better pipelining

* hex-cont/cpy: optimize CPY and CONT ops

The primary change is to avoid scalar divs in the inner loops.
We were calling hvx_copy_uu(... type_size) where type_size is non a constexpr.
This causes runtime divs by that value which is normally just 4 or 2 (f32/f16).

* hex-get-rows: optimize GET_ROWS for large rows

We now use DMA for larger rows and also split them into chunks to improve perf for Qwen3.5 and other models
that do lots of GET_ROWS with huge (2MB+ rows).

Also bump the DMA queue depth now that we can take advantage of it.

* hex-concat: unroll the inner loops of concat_2d

* hex-concat: more updates to concat_2d to improve perf a bit further

* hex-cpy: fixed n_rows per thread checks in the copy ops

* hmx-fa: fix alignment issues while computing dma sizes

* hex-set-rows: add early returns for idle threads

* hvx-rope: minor optimization to replace loops with fastdiv logic

* hex-rope: replace scalar tail processing with HVX

* hex-rope: optimize rope cache init with HVX

Add hvx-utils sin/cos helpers that use an aprox method (similar to rsqrt, inverse, etc)
Use the helpers to optimize ROPE.

This commit is contained in:

Max Krasnyansky

2026-05-26 06:20:05 -07:00

committed by

GitHub

parent 678d43d720

commit ef66bfab68

14 changed files with 867 additions and 230 deletions

									
										ggml/src/ggml-hexagon/htp/htp-ctx.h
									
		+1
		
												View File
												
				@@ -104,6 +104,7 @@ int op_argsort(struct htp_ops_context * octx);

				int op_ssm_conv(struct htp_ops_context * octx);

				int op_cumsum(struct htp_ops_context * octx);

				int op_fill(struct htp_ops_context * octx);

				int op_concat(struct htp_ops_context * octx);

				int op_diag(struct htp_ops_context * octx);

				int op_solve_tri(struct htp_ops_context * octx);

				int op_gated_delta_net(struct htp_ops_context * octx);