Supported Features
What Intel® Inference Microservices supports today, and which capabilities it passes straight through to vLLM for you to configure yourself. Use this page to check an assumption before you design around it.
Everything below describes the published images at tag 0.1.0.
Validated capabilities
These are part of the product surface and covered by the built-in configurations.
| Capability | Status | Where |
|---|---|---|
POST /v1/chat/completions | Supported | API |
POST /v1/completions | Supported | API |
GET /v1/models | Supported — returns one model | API |
GET /health | Supported — 200 ready, 503 not ready | API |
GET /metrics (Prometheus) | Supported | API |
Streaming ("stream": true) | Supported | API |
| OpenAI SDK, LangChain, LlamaIndex, LiteLLM, Haystack | Supported — base URL only | API |
| Automatic hardware detection and configuration | Supported | Meet |
| Kubernetes Deployment + Service with probes | Supported | Deploy on Kubernetes |
| Docker container health check | Supported — built into the image | Quickstart |
| Pre-downloaded model cache | Supported | Model caching |
| Pinning a built-in configuration | Supported — INFERENCE_PROFILE_ID | Environment variables |
| Non-root container, no shell | Supported | Security |
| Model-provided code execution | Granted only where the model requires it; INFERENCE_TRUST_REMOTE_CODE decides explicitly | Security |
vLLM pass-throughs
INFERENCE_ENGINE_ARGS merges a JSON object of vLLM flags over the built-in configuration. That makes the following reachable — but the resulting combination is yours, not the Intel®-validated one for that model, so verify it with dry-run and load-test it before you rely on it.
| Capability | How | Caveat |
|---|---|---|
| Tool / function calling | {"enable-auto-tool-choice": true, "tool-call-parser": "<parser>"} | The parser is model-specific and must match the model's template |
| Structured / JSON output | {"guided-decoding-backend": "xgrammar"} plus per-request response_format | Backend and model support vary |
| Shorter or longer context | {"max-model-len": 4096} | Raising it above the tuned value increases KV-cache memory and can cause OOM |
| Prefix caching | {"enable-prefix-caching": true} | Trades memory for repeated-prompt latency |
| LoRA adapters | {"enable-lora": true, "lora-modules": [...]} | Adapters are your responsibility to mount and validate |
| Reasoning parsers | {"reasoning-parser": "<parser>"} | Model-specific |
A bad value fails in one of two ways, and they look nothing alike:
| What you passed | What happens |
|---|---|
| Malformed JSON, or JSON that is not an object | Ignored with a warning. The container serves with the built-in configuration. |
| Valid JSON, but a flag vLLM rejects | The container fails at startup. vLLM's own parser validates the flags before the engine launches. |
dry-run --format json tells you which one you have. It either prints the resolved flags in "command_script" or fails with the same error serve would give.
Multimodal models
Two catalog entries are tagged multimodal: Gemma E4B Instruct and Qwen3-VL 30B-A3B Instruct. For those images, image inputs use the standard OpenAI content-parts shape that vLLM implements, on the same /v1/chat/completions endpoint — there is no separate endpoint and no Intel®-specific request format. Input types accepted per model follow the model card; see the vLLM multimodal inputs reference for the exact request shape.
Text-only catalog entries do not accept image parts.
Processors and platforms
| Supported | |
|---|---|
| Processors | Intel® Xeon® CPU (xeon): Intel® Xeon® 4, Intel® Xeon® 5, and Intel® Xeon® 6 processors (Sapphire Rapids, Emerald Rapids, Granite Rapids, Sierra Forest) — see Supported Intel® platforms |
| Engine | vLLM (INFERENCE_ENGINE=vllm) |
| Accelerator family | Intel® Xeon® CPU (INFERENCE_ACCELERATOR_TYPE=xeon) |
| Precision | bf16 (auto resolves to the same) |
| Orchestration | Kubernetes 1.24+ — Deployment plus Service |
| Single host | Linux + Docker 20.10+ (Ubuntu 22.04 LTS or later recommended) |
| Where you find image names | Intel® Software Catalog |
Details and sizing: Supported Intel® platforms.
Scaling
One container serves one model, and GET /v1/models returns a single entry. To serve several models, run one Deployment per model and route between them in your application or with a gateway such as LiteLLM. To serve more traffic for one model, add replicas behind the Service. Keep the readiness probe on /health so traffic reaches a pod only after its engine has loaded the model. Watch vllm:num_requests_waiting to decide when to add replicas (API).
Related pages
- API — endpoints, streaming, tool calling, metrics
- Environment variables — every setting, including
INFERENCE_ENGINE_ARGS - FAQ — short answers to common questions
- Glossary — profiles, NUMA, AMX, bf16, tensor parallelism