Qwen/Qwen3.5 9B

• License: Apache-2.0
• Release date: Not documented in the upstream model card.
• Modalities: Input: text, image; Output: text
• Original model: Qwen/Qwen3.5-9B

Description

Qwen3.5-9B is a multilingual, post-trained vision-language model for chat, reasoning, coding, and visual understanding. Its hybrid architecture combines Gated DeltaNet with gated attention for efficient inference, with a native 262,144-token context window.

Run

self serve qwen3.5:9b

Here is a link to the API docs: Text API.

Example Request

With the server running, open another terminal and send a request:

curl http://localhost:8080/v1/chat/completions \
	-H 'Content-Type: application/json' \
	-d '{"model":"qwen3.5:9b","messages":[{"role":"user","content":"Say hello in one word."}],"reasoning_effort":"none","max_tokens":16}'

The server returns an OpenAI-compatible JSON chat completion.

Engine Parameters

Served by upstream llama.cpp llama-server through the llama-server adapter. The model stays loaded between requests; llama.cpp handles its chat template, reasoning split, and sampler.

Parameter	Default	What it controls
context_size	16384	Startup context size in tokens; accepts 512 to 1048576. The model's native limit is 262144.
threads	upstream engine default	Startup CPU workers; accepts 1 to 1024. Not set by this registry entry.
gpu_layers	upstream engine default	Startup GPU offload; accepts -1 to 10000; -1 means all layers. Not set by this registry entry.
flash_attn	upstream engine default	Startup Flash Attention mode: auto, on, or off. Not set by this registry entry.
parallel	upstream engine default	Startup llama-server slots; accepts 1 to 256. Not set by this registry entry.
thinking	on	Default thinking mode: on or off; a request can override it.
temperature	1.0 (registry only)	Accepted setting, range 0 to 2; currently not forwarded as a sampler default. Use the request field.
top_p	0.95 (registry only)	Accepted setting, range 0 to 1; currently not forwarded as a sampler default. Use the request field.
top_k	20 (registry only)	Accepted setting, range 0 to 100000; currently not forwarded to the engine. No public request field.
min_p	0.0 (registry only)	Accepted setting, range 0 to 1; currently not forwarded to the engine. No public request field.
presence_penalty	1.5 (registry only)	Accepted setting, range -2 to 2; currently not forwarded as a sampler default. Use the request field.

Override startup settings with --set <parameter>=<value>. Per-request API fields are JSON properties, not startup flags. Registry-only sampling values describe recommendations, not effective engine defaults in the current adapter; omitted request values fall back to llama-server's sampler.

Quantizations and Files

Default variant: q4. Sizes and SHA-256 hashes are pinned in the model registry.

q4 (default)

File	Role	Size	SHA-256	Origin
Qwen3.5-9B-Q4_K_M.gguf	model	5.29 GiB (5680522464 bytes)	03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8	unsloth/Qwen3.5-9B-GGUF
mmproj-F16.gguf	mmproj	875.6 MiB (918166080 bytes)	f70dc3509053962b0d0d3ee8a7eacebf5d60aa560cad78254ae8698516ae029f	unsloth/Qwen3.5-9B-GGUF

Total download: 6.15 GiB (6598688544 bytes).

q8

File	Role	Size	SHA-256	Origin
Qwen3.5-9B-Q8_0.gguf	model	8.87 GiB (9527502048 bytes)	809626574d0cb43d4becfa56169980da2bb448f2299270f7be443cb89d0a6ae4	unsloth/Qwen3.5-9B-GGUF
mmproj-F16.gguf	mmproj	875.6 MiB (918166080 bytes)	f70dc3509053962b0d0d3ee8a7eacebf5d60aa560cad78254ae8698516ae029f	unsloth/Qwen3.5-9B-GGUF

Total download: 9.73 GiB (10445668128 bytes).

Notes

• Thinking is on by default. Output reasoning appears in reasoning_content, separate from content. Disable it per request with "reasoning_effort":"none" or "chat_template_kwargs":{"enable_thinking":false}, or at startup with --set thinking=off.
• The upstream card recommends general thinking-task sampling of temperature 1.0, top_p 0.95, top_k 20, min_p 0, and presence_penalty 1.5. For non-thinking tasks it recommends temperature 0.7 and top_p 0.8. Set the supported request fields explicitly; the registry carries these recommendations but does not apply them through the current adapter.
• Image input uses user-message image_url content parts with an HTTPS URL or base64 data URL. The current adapter accepts one image per request, despite the registry's multi-image capability label. Video input is not exposed by this API.
• The default context is 16,384 tokens, below the model's native 262,144. Download size is not the total runtime memory requirement.
• Origin links use main because these registry entries pin hashes, not repository revisions.

Quality benchmarks

MMLU-Pro: Publisher-reported

Qwen/Qwen3.5-9B

Publisher-reported Qwen3.5-9B column, Language evaluation. Not a local quantization result or independent rank.

Dataset	Metric	Score
MMLU-Pro	Accuracy (%)	82.50

Benchmarks

No submitted benchmark reports yet.