Google/Gemma 4 E4B

• License: Apache-2.0
• Release date: Not documented in the upstream model card.
• Modalities: Input: text, image (served); Output: text
• Original model: google/gemma-4-E4B-it

Description

Gemma 4 E4B is Google DeepMind's instruction-tuned multilingual multimodal model for reasoning, conversation, and visual understanding. It uses per-layer embeddings to support efficient on-device deployment: 4.5B effective parameters, 8B including embeddings, and a 131,072-token context window. Upstream also supports audio input, which is not exposed by this server's text adapter.

Run

self serve gemma4:4b

Here is a link to the API docs: Text API.

Example Request

With the server running, open another terminal and send a request:

curl http://localhost:8080/v1/chat/completions \
	-H 'Content-Type: application/json' \
	-d '{"model":"gemma4:4b","messages":[{"role":"user","content":"Say hello in one word."}],"max_tokens":16}'

The server returns an OpenAI-compatible JSON chat completion.

Engine Parameters

Served by upstream llama.cpp llama-server through the llama-server adapter. The model stays loaded between requests; llama.cpp handles its chat template, reasoning split, and sampler.

Parameter	Default	What it controls
context_size	16384	Startup context size in tokens; accepts 512 to 1048576. The model's native limit is 131072.
threads	upstream engine default	Startup CPU workers; accepts 1 to 1024. Not set by this registry entry.
gpu_layers	upstream engine default	Startup GPU offload; accepts -1 to 10000; -1 means all layers. Not set by this registry entry.
flash_attn	upstream engine default	Startup Flash Attention mode: auto, on, or off. Not set by this registry entry.
parallel	upstream engine default	Startup llama-server slots; accepts 1 to 256. Not set by this registry entry.
thinking	off	Default thinking mode: on or off; a request can override it.
temperature	1.0 (registry only)	Accepted setting, range 0 to 2; currently not forwarded as a sampler default. Use the request field.
top_p	0.95 (registry only)	Accepted setting, range 0 to 1; currently not forwarded as a sampler default. Use the request field.
top_k	64 (registry only)	Accepted setting, range 0 to 100000; currently not forwarded to the engine. No public request field.
min_p	0.0 (registry only)	Accepted setting, range 0 to 1; currently not forwarded to the engine. No public request field.
presence_penalty	unset; upstream engine default	Accepted setting, range -2 to 2; currently not forwarded as a sampler default. Use the request field.

Override startup settings with --set <parameter>=<value>. Per-request API fields are JSON properties, not startup flags. Registry-only sampling values describe recommendations, not effective engine defaults in the current adapter; omitted request values fall back to llama-server's sampler.

Quantizations and Files

Default variant: q4. Sizes and SHA-256 hashes are pinned in the model registry.

q4 (default)

File	Role	Size	SHA-256	Origin
gemma-4-E4B_q4_0-it.gguf	model	4.80 GiB (5154941280 bytes)	676c35070db6dbe52f93e9c864ee0fba4eddea94b9c875d9cb10daff453fbaee	google/gemma-4-E4B-it-qat-q4_0-gguf
gemma-4-E4B-it-mmproj.gguf	mmproj	945.6 MiB (991552256 bytes)	7498a37cb619e55f2fcf87eb931f56e99389ed6d432e4c5c66110694c0d65578	google/gemma-4-E4B-it-qat-q4_0-gguf

Total download: 5.72 GiB (6146493536 bytes).

q8

File	Role	Size	SHA-256	Origin
gemma-4-E4B-it-Q8_0.gguf	model	7.48 GiB (8031242688 bytes)	34be82b17b4942d389b9b527170c4b058027abdd32531fda063d3d97dd8ce80a	ggml-org/gemma-4-E4B-it-GGUF
mmproj-gemma-4-E4B-it-Q8_0.gguf	mmproj	533.9 MiB (559874816 bytes)	197f49a93027f9843772bd24a6a9e0be2a32a788de5a3def330e9c585d86edd1	ggml-org/gemma-4-E4B-it-GGUF

Total download: 8.00 GiB (8591117504 bytes).

Notes

• Thinking is off by default. Enable it per request with "reasoning_effort":"low" (any supported level except none) or "chat_template_kwargs":{"enable_thinking":true}, or at startup with --set thinking=on. Reasoning appears in reasoning_content, separate from content.
• The upstream card recommends temperature 1.0, top_p 0.95, and top_k 64. Set supported request fields explicitly; the registry records these recommendations but the current adapter does not forward sampling defaults. Do not include earlier reasoning in conversation history; use final answers only.
• q4 uses Google's quantization-aware-trained Q4_0 weights; q8 uses ggml-org's Q8_0 weights. Both include their own required vision projector.
• Image input uses user-message image_url content parts with an HTTPS URL or base64 data URL; upstream recommends placing the image before the text. The current adapter accepts one image per request, despite the registry's multi-image capability label. Audio and video input are not exposed by this API; the audio encoder is not used.
• The default context is 16,384 tokens, below the model's native 131,072. Download size is not the total runtime memory requirement.
• Origin links use main because these registry entries pin hashes, not repository revisions.

Quality benchmarks

MMLU-Pro: Publisher-reported

google/gemma-4-E4B-it

Publisher-reported instruction-tuned Gemma 4 E4B column. Not a local quantization result or independent rank.

Dataset	Metric	Score
MMLU-Pro	Accuracy (%)	69.40

Benchmarks

No submitted benchmark reports yet.