Local deployment

Local and edge runtimes convert or load HF-related artefacts for laptops, Apple Silicon and mobile.

Adapted for this playbook from the 🤗 Transformers documentation by Hugging Face. Official page: https://huggingface.co/docs/transformers/community_integrations/llama_cpp. Images © Hugging Face (documentation-images) unless noted. This is not a substitute for the upstream docs — verify against the current version.

Covers Hugging Face pages: community_integrations/llama_cpp, mlx, candle, executorch

Runtime	Strength	Doc
llama.cpp	CPU/GPU local LLMs, GGUF	llama.cpp
MLX	Apple Silicon	MLX
Candle	Rust	Candle
ExecuTorch	On-device PyTorch	ExecuTorch

Pair with GGUF quantization for llama.cpp workflows.

Key takeaways

• Local ≠ same quality/latency as data-centre serving — size models accordingly.
• Track conversion scripts and quantization levels in your SBOM.

What to do next

Fine-tuning stack.