Hardware

Pick silicon that matches your training/inference profile — and budget for memory, not only FLOPs.

Adapted for this playbook from the 🤗 Transformers documentation by Hugging Face. Official page: https://huggingface.co/docs/transformers/perf_hardware. Images © Hugging Face (documentation-images) unless noted. This is not a substitute for the upstream docs — verify against the current version.

Covers Hugging Face pages: perf_hardware, perf_train_cpu, perf_train_special, perf_train_gaudi, model_memory_anatomy

Guidance map

Platform	Official doc
GPU workstation	Building a GPU workstation
CPU	CPU
Apple Silicon	Apple Silicon
Intel Gaudi	Gaudi

Rules of thumb

• Prefer fewer faster GPUs with enough HBM over many under-memoried cards for LLM fine-tuning
• Account for activation memory at long context
• Inference and training sizing differ — see also Model FinOps

Key takeaways

• Memory headroom beats peak TFLOPs for most fine-tunes.
• Keep a documented SKU matrix for the team.

What to do next

Quantization concepts.