← Docs index

Do this once, before the first docker run. Then go to the Quickstart.

Hardware

Suggested minimum for 4B–8B models (including the Quickstart). Larger models need more — see Supported Intel® platforms.

32 GB is a floor. Bigger models may need additional RAM.

To check that this machine is a supported Intel® Xeon® CPU, run detect-hardware inside the container. It does not load a model or start the server:

docker run --rm --cap-add SYS_NICE intel/inference-xeon-base:0.1.0 detect-hardware

This host is supported when the output includes:

• Family: xeon — Intel® Xeon® CPU
• Model: one of xeon_spr, xeon_emr, xeon_gnr, or xeon_srf — Intel® Xeon® 4, 5, or 6 processor

Anything else is not a supported generation for this release. Sample output: Supported Intel® platforms.

Software

Nothing else. No Python environment, no vLLM install — those are inside the image.

1. Install and configure Docker

Follow the Docker Engine install guide for your Linux distribution.

Then add your user to the docker group so you can run docker without sudo:

sudo groupadd docker
sudo usermod -aG docker $USER

If groupadd says the group already exists, that is fine. Log out and back in (or reboot) so the new group takes effect. Confirm:

docker run hello-world

More detail: Linux post-install.

2. Log in to Docker

Images live under the intel/ namespace. Log in so pulls are authenticated (and so you are not limited by anonymous Docker Hub rate limits):

docker login

If the Intel® Software Catalog listing for your image uses a private registry, log in to that registry instead, with the credentials from the listing.

3. Hugging Face: token and model approval

A gated model is one Hugging Face will not download until you accept its license on the model page. The Quickstart example (Qwen3-4B) is not gated. Llama and Gemma are. If the listing's docker run has no -e HF_TOKEN, skip this section.

1. Create a token with Read access: Hugging Face access tokens.
2. Open the model page on Hugging Face and accept the license (Agree and access repository). A valid token still returns 401 if you have not approved that model.
3. Pass it when you run a gated image. Paste the listing's docker run from the Intel® Software Catalog — gated listings already include -e HF_TOKEN:

docker run --rm -p 8000:8000 \
  --cap-add SYS_NICE \
  --shm-size=2g \
  -e HF_TOKEN=<your-token> \
  -v ~/.cache/huggingface:/workspace/model-cache/huggingface \
  intel/inference-xeon-meta-llama-llama-3.2-3b-instruct:0.1.0

A named example (Llama 3.2 3B Instruct) is in Quickstart. These docs assume Linux (bash/zsh). Putting the token on the command line writes it to shell history — how to pass it from a file or a Kubernetes secret: Security.

Next

Quickstart — serve a model.

Related pages

• Supported Intel® platforms — processor generations and memory sizing
• Model catalog — validated model Docker images
• Security — passing HF_TOKEN without leaking it
• Troubleshooting — pull access denied, Hugging Face 401