Mistral

Mistral models are efficient decoder-only LLMs widely used in production open-weight deployments. Transformers documents attention-mask behaviour and SDPA/FlashAttention support.

Adapted for this playbook from the 🤗 Transformers documentation by Hugging Face. Official page: https://huggingface.co/docs/transformers/model_doc/mistral. Images © Hugging Face (documentation-images) unless noted. This is not a substitute for the upstream docs — verify against the current version.

Covers Hugging Face pages: model_doc/mistral

Source: Hugging Face Transformers documentation (mistral-attn-mask.png).

Executive summary

Choose Mistral-family instruct models when you want strong quality-per-compute and mature Hub/tooling support. Always apply the model’s chat template.

Quick usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-Instruct-v0.2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", attn_implementation="sdpa")

messages = [{"role": "user", "content": "Explain sliding window attention briefly."}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128)
print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Official API

Mistral model doc.

What to do next

Whisper.