Embeddings API

Embedding models expose the official OpenAI-compatible embeddings endpoint. The server runs one selected embedding model per process.

Health, model metadata and the error format are shared by all modalities; see the API overview.

Endpoint

POST /v1/embeddings

Models and quick start

Browse embedding models

self serve nomic-embed-text-v1.5:137m

POST /v1/embeddings

This is the OpenAI-compatible embeddings endpoint. Requests must use Content-Type: application/json; unknown top-level fields are rejected. The route accepts one string or a non-empty batch of strings and content objects. Unknown fields are rejected at every level. Content objects follow llama.cpp's multimodal embedding extension; they are not part of the standard OpenAI text API.

Request body

inputstring | array of strings or content objectsrequired

Text to embed, or a batch of {content: [...]} objects. Each content part is {type: "text", text: "..."}, {type: "image_url", image_url: {url: "data:image/png;base64,..."}}, or {type: "input_audio", input_audio: {data: "...", format: "wav"}}. Audio data accepts raw base64 or an audio/wav base64 data URL. Images must be PNG or JPEG data URLs or remote HTTPS URLs, using the same image_url content part as the chat API. Media requires the corresponding loaded-model capability.

Media is limited to 10 MiB decoded per part; the JSON body limit is 32 MiB. Images are limited to 16 megapixels and 8192 pixels per side. Audio must be PCM16 WAV, mono or stereo, 8-96 kHz. Local file URLs are rejected. Remote images use the same guarded fetcher as decision and chat models: HTTPS and public destinations only by default, a 15-second fetch timeout, and at most three redirects. Use --allow-http-images to permit plain HTTP, and --allow-private-images to permit private or loopback destinations. These flags weaken the default network restrictions and should only be enabled for trusted clients. Audio remains inline-only. Empty text, empty content, malformed media and mismatched MIME types return 400. Unsupported content types or model modalities return 422 before any inference.

modelstringoptional

Accepted and ignored: a server runs the model selected at startup.

encoding_formatstringoptional

Use float or omit the field. Other formats are not supported.

dimensionsintegeroptional

Dimensionality changes are not supported by the loaded model and return 422 unsupported_capability.

userstringoptional

Optional end-user identifier. It is accepted and ignored.

multi_embeddingbooleanoptional

Request a matrix of token vectors for each input. Defaults to false. Requires the loaded model's multi-embedding output capability and a compatible adapter; otherwise the request returns 422 unsupported_capability before inference.

input_typequery | documentoptional

Select query or document encoding for multi-vector retrieval. Requires multi_embedding: true; defaults to document in that mode.

Multi-vector embeddings

multi-embedding is an additive capability for late-interaction retrievers, including PPLX Embed v2 Late 0.6B and 9B. It means multiple vectors per input, not a batch of single-vector inputs. Model metadata exposes the capability under capabilities.output in GET /v1/model.

For multi_embedding: true, each data item has object: "multi-embedding" and an embedding matrix: [token][dimension]. A batch still has one data item per input. Query and document inputs use separate encoding calls. These fields and the matrix response are extensions to the standard OpenAI API.

PPLX's two sizes produce 128-dimensional token vectors in a shared embedding space. Retrieval uses MaxSim, not pooled cosine similarity. The existing /similarity endpoint remains a single-vector cosine operation. Multi-vector-only models reject that operation and ordinary single-vector embedding calls rather than silently pooling their vectors. Neither PPLX checkpoint is registered yet; the capability contract does not supply its inference runtime or converted weights.

Example request

curl

curl http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"nomic-embed-text-v1.5:137m","input":"Represent this sentence."}'

Python

import requests

response = requests.post(
    "http://localhost:8080/v1/embeddings",
    json={"model": "nomic-embed-text-v1.5:137m", "input": "Represent this sentence."},
    timeout=60,
)
response.raise_for_status()
print(response.json())

JavaScript

const response = await fetch('http://localhost:8080/v1/embeddings', {
  method: 'POST',
  headers: { 'content-type': 'application/json' },
  body: JSON.stringify({ model: 'nomic-embed-text-v1.5:137m', input: 'Represent this sentence.' })
})

if (!response.ok) throw new Error(await response.text())
console.log(await response.json())

Response

{
  "object": "list",
  "data": [{"object": "embedding", "embedding": [0.0123, -0.0456], "index": 0}],
  "model": "nomic-embed-text-v1.5:137m",
  "usage": {"prompt_tokens": 4, "total_tokens": 4, "latency_ms": 12.4}
}

Response body

objectstring

list.

dataarray

Embedding objects with object: "embedding", integer index, and a numeric embedding array.

modelstring

Model used to create the embeddings.

usageobject

Token counts returned by llama.cpp: prompt_tokens and total_tokens. For a batch request, they cover all supplied input strings. latency_ms is the total time spent handling the request, including embedding inference.

Embedding vectors are returned as floating-point arrays. The vector dimension depends on the loaded model.

Error behavior

Malformed JSON, missing input, empty batches, wrong content types, and unknown fields return 400 invalid_request. Unsupported encoding_format values and dimensions requests return 422 unsupported_capability.

{
  "error": {
    "type": "unsupported_capability",
    "message": "dimensions are not supported by the loaded embedding model"
  }
}

Multimodal Examples

Remote image input uses the same image content part as chat completions:

{
  "input": [{"content": [
    {"type": "image_url", "image_url": {"url": "https://example.com/picture.png"}}
  ]}]
}

Start the multimodal model (the pinned projector downloads automatically):

self serve embeddinggemma-2:740m

The following Node.js example sends a text scalar, a text/image object, and an audio object in one batch. Replace the two filenames with your own media files.

import { readFileSync } from 'node:fs'

const image = `data:image/png;base64,${readFileSync('picture.png').toString('base64')}`
const audio = readFileSync('recording.wav').toString('base64')
const input = [
  'A bird singing in a tree.',
  { content: [
    { type: 'text', text: 'A bird in a tree.' },
    { type: 'image_url', image_url: { url: image } }
  ] },
  { content: [{ type: 'input_audio', input_audio: { data: audio, format: 'wav' } }] }
]
const response = await fetch('http://localhost:8080/v1/embeddings', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ input })
})
if (!response.ok) throw new Error(await response.text())
const result = await response.json()
console.log(result.data.map(item => ({ index: item.index, dimensions: item.embedding.length })))

EmbeddingGemma 2 returns full 768-dimensional vectors. Text-only scalar and string-batch requests remain compatible. /v1/models reports the actual declared input capabilities (text, vision, audio for this model).

Multimodal similarity uses a single string or content object as input and a non-empty array of strings or content objects as ref:

const response = await fetch('http://localhost:8080/similarity', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    input: { content: [{ type: 'image_url', image_url: { url: image } }] },
    ref: ['A bird in a tree.', { content: [{ type: 'input_audio', input_audio: { data: audio, format: 'wav' } }] }]
  })
})
if (!response.ok) throw new Error(await response.text())
console.log((await response.json()).similarities)

POST /similarity

Compute normalized cosine similarity between input and every item in ref. Scores are in the range [0, 1] and have the same order as the supplied ref array.

Request body

inputstring | content objectrequired

The text or multimodal content object to compare against.

refarray of strings or content objectsrequired

One or more text or multimodal items to compare with input.

Example request

curl http://localhost:8080/similarity \
  -H 'Content-Type: application/json' \
  -d '{"input":"A cat is sleeping.","ref":["A kitten is asleep.","The weather is sunny."]}'

Response

{
  "similarities": [0.91, 0.08],
  "usage": {"prompt_tokens": 10, "total_tokens": 10, "latency_ms": 24.7}
}

similarities contains one score per reference string. A score of 1 means identical vector direction, 0.5 means orthogonal vectors, and 0 means opposite direction. Scores are normalized from cosine similarity with (cosine + 1) / 2; they are measured between input and each ref independently, so they do not imply a global similarity across all references or add up to 1. usage.latency_ms is the total time spent handling the request, including all embedding calls. Missing or empty fields, empty reference strings, unknown fields, mismatched embedding dimensions, and zero-magnitude vectors return an error.