Fine-tuning stack

Beyond Trainer, the ecosystem provides opinionated fine-tuning frameworks that still rely on Transformers definitions.

Adapted for this playbook from the 🤗 Transformers documentation by Hugging Face. Official page: https://huggingface.co/docs/transformers/community_integrations/trl. Images © Hugging Face (documentation-images) unless noted. This is not a substitute for the upstream docs — verify against the current version.

Covers Hugging Face pages: community_integrations/trl, unsloth, axolotl, nemo_automodel_finetuning

Tool	Focus	Doc
TRL	SFT, reward modelling, RLHF/DPO-style methods	TRL
Unsloth	Fast / memory-efficient LoRA fine-tunes	Unsloth
Axolotl	YAML-driven fine-tune configs	Axolotl
NeMo Automodel	NVIDIA fine-tune workflows	NeMo Automodel

Suggested path

1. Prove data quality with a small Trainer/TRL SFT run
2. Move to Unsloth/Axolotl for faster iteration if needed
3. Evaluate with held-out sets and safety suites before merge/deploy

What to do next

Pretraining stack.