← Docs index

The Intel® Inference Microservices container accepts five commands and nothing else: serve (the default), dry-run, list-profiles, download-to-cache, and detect-hardware. That is the whole command surface — there is no other CLI. Everything configurable is an INFERENCE_* environment variable (-e on docker run, env: on Kubernetes), not a flag on the command.

You only need serve. It is the default: pass no command and the container serves the model. That is every docker run in the Quickstart.

The other four are optional:

• download-to-cache downloads weights ahead of time, so a later serve does not have to (Model caching).
• dry-run, list-profiles, and detect-hardware only read. They load no weights and bind no port, so they are safe to run while another container is serving.

Each command exports IMAGE. That is the last argument of the catalog listing's docker run (the image name, keep the tag), not the whole command. Do not type the name by hand.

export IMAGE=<intel/inference-...:tag>

docker run --rm --cap-add SYS_NICE "$IMAGE" --help

  Detect the host accelerator, pick a matching serving profile, and run or
  inspect the engine it resolves to.

Options:
  --help  Show this message and exit.

Commands:
  detect-hardware    Probe the accelerator family, model, and available...
  download-to-cache  Pull model weights into the local cache so a later...
  dry-run            Resolve a profile and print the engine command it...
  list-profiles      List every discovered profile for the configured...
  serve              Resolve a profile for the configured model and exec...

Every command also takes --help. Names are one token: list-profiles, not list profiles.

serve

What it is: the production command. Detect the host, pick a built-in config for the model, then replace the process with vLLM. After this, port 8000 speaks /v1/chat/completions.

When to use it: when the process should actually serve. On Kubernetes the Deployment runs serve by default (no extra args). A local Quickstart is the same command with no name.

--cap-add SYS_NICE lets the engine pin memory to the CPU socket it is using. --shm-size=2g gives the engine enough shared memory to start (Docker's default 64 MiB is not). -v is model caching.

export IMAGE=<intel/inference-...:tag>

docker run --rm -p 8000:8000 \
  --cap-add SYS_NICE \
  --shm-size=2g \
  -v ~/.cache/huggingface:/workspace/model-cache/huggingface \
  "$IMAGE"

Same thing, named:

export IMAGE=<intel/inference-...:tag>

docker run --rm -p 8000:8000 \
  --cap-add SYS_NICE \
  --shm-size=2g \
  "$IMAGE" \
  serve

A catalog model image already has the id baked in, so the commands above are enough. On the base image you must also pass -e INFERENCE_MODEL_ID="$MODEL_ID".

Flags: none. Port, token, log level: Environment variables. Gated models need HF_TOKEN — the listing's docker run already includes it when the model is gated.

dry-run

What it is: serve without starting the engine. It still detects hardware and picks a config, then prints that config and the vLLM command it would exec.

When to use it: before a rollout, or when you want to see what would run without waiting for weights.

export IMAGE=<intel/inference-...:tag>

docker run --rm --cap-add SYS_NICE "$IMAGE" dry-run --format json

Example (--format json):

{
  "profile_id": "vllm-xeon-bf16-tp1",
  "model_id": "<MODEL_ID>",
  "detection": {
    "accelerator_family": "xeon",
    "accelerator_model": "xeon_gnr",
    "accelerator_count": 16
  },
  "trust_remote_code": false,
  "command_script": "#!/bin/bash\nexport VLLM_DO_NOT_TRACK=1\npython -m vllm.entrypoints.openai.api_server --model <MODEL_ID> ..."
}

Safe to run while another container is already serving on port 8000 — this command does not bind the port.

list-profiles

What it is: a read-only listing of configs baked into this image, marked compatible or not for the host you are on.

When to use it: “what can this box run?” [primary] is the recommended row. numa_mismatch means that row wants more NUMA nodes than you have. serve picks a compatible row on its own.

The name is list-profiles (hyphen). list profiles fails with No such command 'list'.

export IMAGE=<intel/inference-...:tag>

docker run --rm --cap-add SYS_NICE "$IMAGE" list-profiles

Compatible profiles:
  [+] vllm-xeon-bf16-tp1 (optimized, bf16, tp1) [primary]
  [+] vllm-xeon-bf16 (general, bf16, tp1)
Incompatible profiles:
  [-] vllm-xeon-bf16-tp2 (optimized, bf16, tp2) - numa_mismatch
  [-] vllm-xeon-bf16-tp4 (optimized, bf16, tp4) - numa_mismatch

download-to-cache

What it is: fetch model files into a directory now, so a later serve does not block on Hugging Face.

When to use it: production or CI — warm a volume, then start serve with that same -v mount. See Caching.

export IMAGE=<intel/inference-...:tag>

docker run --rm \
  --cap-add SYS_NICE \
  -v /data/cache:/workspace/model-cache \
  "$IMAGE" \
  download-to-cache

Gated models still need -e HF_TOKEN=.... Skip it when the listing's docker run has none.

detect-hardware

What it is: a probe of this machine only — family, model, cores, NUMA, AMX / AVX-512 / AVX2 / VNNI. It does not pick a config and does not start the server.

When to use it: confirm the container sees the Intel® processor (Intel® Xeon® CPU) you think it does, before you file a “wrong config” bug, or to label Kubernetes nodes (--format label).

This command does not load a model, so use the base image — not a catalog model image. It does not start the server.

docker run --rm --cap-add SYS_NICE intel/inference-xeon-base:0.1.0 detect-hardware

Family:       xeon
Model:        xeon_gnr
Cores:        16
Node Capacity: 192
NUMA Nodes:   2
Cpuset Bind:  0-15,64-79
Capabilities: amx=True, avx512=True, avx2=True, vnni=True

docker run --rm --cap-add SYS_NICE intel/inference-xeon-base:0.1.0 detect-hardware --format json --verbose

Cores is the cpuset the container was given; Node Capacity is the whole host. When those differ, the container is restricted and Cores is the number that drives configuration selection. Family: xeon means Intel® Xeon® CPU. What the Model: values mean: Glossary.

Related pages

• Environment variables — everything you configure, since none of it is a CLI flag
• Model caching — the download-to-cache workflow end to end
• Glossary — profiles, numa_mismatch, [primary], accelerator models
• Troubleshooting — using these commands to diagnose a failed start