Serving

Transformers includes serving paths (including Serve CLI) so you can expose models locally and connect chat UIs.

Adapted for this playbook from the 🤗 Transformers documentation by Hugging Face. Official page: https://huggingface.co/docs/transformers/serve-cli/serving. Images © Hugging Face (documentation-images) unless noted. This is not a substitute for the upstream docs — verify against the current version.

Covers Hugging Face pages: serve-cli/serving, serve-cli/serving_optims, serve-cli/cursor, serve-cli/jan, serve-cli/openweb_ui, serve-cli/tiny_agents

Executive summary

Use Serve CLI for local/OpenAI-compatible experiments. For production QPS, evaluate vLLM/SGLang/TGI and put auth, quotas and observability in front.

Concepts

• Start a server that speaks a familiar chat completions API
• Point UIs (Open WebUI, Jan) or IDEs (Cursor) at the endpoint
• Enable server-side optimisations documented by Hugging Face

Follow the live commands in Serve CLI and Server optimisations.

Client guides (official)

Client	Doc
Cursor	serve-cli/cursor
Jan	serve-cli/jan
Open WebUI	serve-cli/openweb_ui
tiny-agents	serve-cli/tiny_agents

Enterprise notes

• Bind to localhost or a private network; put SSO/API keys at the gateway
• Separate “lab serving” from “production serving” SLOs
• Prefer engine-backed deployments for multi-tenant LLMs — Inference engines

Key takeaways

• Serve CLI is excellent for local developer loops.
• Production still needs an inference engine + platform controls.

What to do next

Trainer or Inference engines.