Google/Gemma 4 12B

• License: Apache-2.0
• Release date: Not documented in the upstream model card.
• Modalities: Input: text, image (served); Output: text
• Original model: google/gemma-4-12B-it

Description

Gemma 4 12B Unified is Google DeepMind's instruction-tuned multilingual multimodal model with 11.95B parameters and a 262,144-token context window. Its encoder-free design projects image patches and audio waveforms directly into a shared decoder, reducing multimodal latency for local deployment. Upstream supports text, image, audio, and video understanding; this server exposes text and image input only.

Run

self serve gemma4:12b

Here is a link to the API docs: Text API.

Example Request

With the server running, open another terminal and send a request:

curl http://localhost:8080/v1/chat/completions \
	-H 'Content-Type: application/json' \
	-d '{"model":"gemma4:12b","messages":[{"role":"user","content":"Say hello in one word."}],"max_tokens":16}'

The server returns an OpenAI-compatible JSON chat completion.

Engine Parameters

Served by upstream llama.cpp llama-server through the llama-server adapter. The model stays loaded between requests; llama.cpp handles its chat template, reasoning split, and sampler.

Parameter	Default	What it controls
context_size	16384	Startup context size in tokens; accepts 512 to 1048576. The model's native limit is 262144.
threads	upstream engine default	Startup CPU workers; accepts 1 to 1024. Not set by this registry entry.
gpu_layers	upstream engine default	Startup GPU offload; accepts -1 to 10000; -1 means all layers. Not set by this registry entry.
flash_attn	upstream engine default	Startup Flash Attention mode: auto, on, or off. Not set by this registry entry.
parallel	upstream engine default	Startup llama-server slots; accepts 1 to 256. Not set by this registry entry.
thinking	off	Default thinking mode: on or off; a request can override it.
temperature	1.0 (registry only)	Accepted setting, range 0 to 2; currently not forwarded as a sampler default. Use the request field.
top_p	0.95 (registry only)	Accepted setting, range 0 to 1; currently not forwarded as a sampler default. Use the request field.
top_k	64 (registry only)	Accepted setting, range 0 to 100000; currently not forwarded to the engine. No public request field.
min_p	0.0 (registry only)	Accepted setting, range 0 to 1; currently not forwarded to the engine. No public request field.
presence_penalty	unset; upstream engine default	Accepted setting, range -2 to 2; currently not forwarded as a sampler default. Use the request field.

Override startup settings with --set <parameter>=<value>. Per-request API fields are JSON properties, not startup flags. Registry-only sampling values describe recommendations, not effective engine defaults in the current adapter; omitted request values fall back to llama-server's sampler.

Quantizations and Files

Default variant: q4. Sizes and SHA-256 hashes are pinned in the model registry.

q4 (default)

File	Role	Size	SHA-256	Origin
gemma-4-12b-it-qat-q4_0.gguf	model	6.50 GiB (6975879296 bytes)	93567e57a8fe10b23569b9d9ec38cd005deedf71e29477c421a4b83f418a538b	google/gemma-4-12B-it-qat-q4_0-gguf
mmproj-gemma-4-12b-it-qat-q4_0.gguf	mmproj	167.0 MiB (175115616 bytes)	cb018338a7538a9814d994bfe54644c71eb7ed54e31eae2f721e45fd3c260da7	google/gemma-4-12B-it-qat-q4_0-gguf

Total download: 6.66 GiB (7150994912 bytes).

q8

File	Role	Size	SHA-256	Origin
gemma-4-12B-it-Q8_0.gguf	model	11.80 GiB (12669646976 bytes)	abfc3044b93795f6a446fc1a04468ac2d5fc5e9c92e237f09a5b3384c5a433e4	ggml-org/gemma-4-12B-it-GGUF
mmproj-gemma-4-12B-it-Q8_0.gguf	mmproj	151.6 MiB (158987616 bytes)	59e62255435dda870e2d1de97cc031330b31a898bac12b38a182cecff9cd3738	ggml-org/gemma-4-12B-it-GGUF

Total download: 11.95 GiB (12828634592 bytes).

Notes

• Thinking is off by default. Enable it per request with "reasoning_effort":"low" (any supported level except none) or "chat_template_kwargs":{"enable_thinking":true}, or at startup with --set thinking=on. Reasoning appears in reasoning_content, separate from content.
• The upstream card recommends temperature 1.0, top_p 0.95, and top_k 64. Set supported request fields explicitly; the registry records these recommendations but the current adapter does not forward sampling defaults. Do not include earlier reasoning in conversation history; use final answers only.
• q4 uses Google's quantization-aware-trained Q4_0 weights; q8 uses ggml-org's Q8_0 weights. Both include their own required image projector, even though the original model has no separate vision encoder.
• Image input uses user-message image_url content parts with an HTTPS URL or base64 data URL; upstream recommends placing the image before the text. The current adapter accepts one image per request, despite the registry's multi-image capability label. Audio and video input are not exposed by this API.
• The default context is 16,384 tokens, below the model's native 262,144. Download size is not the total runtime memory requirement.
• Origin links use main because these registry entries pin hashes, not repository revisions.

Quality benchmarks

MMLU-Pro: Publisher-reported

google/gemma-4-12B-it

Publisher-reported instruction-tuned Gemma 4 12B Unified column. Not a local quantization result or independent rank.

Dataset	Metric	Score
MMLU-Pro	Accuracy (%)	77.20

Benchmarks

No submitted benchmark reports yet.