Mistral: Architecture, Training and Deployment

Mistral AI's current open-weight line spans two design points: Mistral Small 4, an Apache 2.0-licensed 119B-total / ~6B-active Mixture-of-Experts model with 128 experts and a 256K context window, and Mistral Medium 3.5, a 128B dense model with the same 256K context. Small 4 is notable for unifying capabilities previously split across separate Mistral product lines — Magistral (reasoning), Pixtral (vision) and Devstral (coding) — into one checkpoint.

Executive summary

Mistral's 2026 open-weight strategy is consolidation: rather than shipping separate reasoning, vision and coding specialist models, Small 4 folds all three into a single Apache 2.0 MoE checkpoint with a configurable reasoning_effort parameter. At roughly 119B total parameters and only ~6B active per token, it is deliberately sized to be self-hostable on a modest multi-GPU footprint (Mistral states as few as four H100-class GPUs), making it one of the more approachable frontier-adjacent open-weight models in this section.

When to use this page

• Choosing a single self-hostable checkpoint that covers reasoning, vision and coding rather than three separate specialist models.
• Sizing minimum GPU infrastructure for a Mistral deployment (Small 4 vs Medium 3.5).
• Understanding the reasoning_effort parameter and when to use none vs high.
• Pair with Open source models, the AI model landscape and Gemma 4 for a comparable "efficient, deployable" design philosophy.

---

1. What kind of model is Mistral Small 4?

Open-weight (Apache 2.0), decoder-based, natively multimodal sparse MoE model unifying reasoning, vision and coding in one checkpoint.

1.1 Apache 2.0

Mistral Small 4 ships under Apache 2.0 — the same permissive baseline as Qwen 3.5, Gemma 4 and GPT-OSS, and materially more commercially friendly than Llama 4's custom Community Licence. There is no field-of-use restriction, monthly-active-user threshold, or attribution clause beyond standard notice preservation.

1.2 Consolidating three product lines into one model

Prior to Small 4, Mistral shipped separate specialist lines:

Prior line	Specialisation
Magistral	Reasoning
Pixtral	Vision
Devstral	Coding agents

Mistral Small 4 unifies these into a single checkpoint rather than requiring an application to route between three separately hosted models depending on task type. This is a meaningful operational simplification: one deployment, one context budget, one set of serving-engine flags, instead of maintaining three separate inference stacks and a router between them.

1.3 Two-model line-up

Variant	Architecture	Total params	Active params	Context
Mistral Small 4	Sparse MoE	119B	~6B (8B including embeddings/output)	256,000
Mistral Medium 3.5	Dense	128B	128B	256,000

Small 4's active-parameter figure is commonly cited two ways: ~6B for the expert/attention compute path alone, or ~8B when embedding and output-projection layers (which run on every token regardless of MoE routing) are included — both numbers describe the same model, just with a different accounting boundary.

---

2. Model configuration

Component	Mistral Small 4
Architecture	Sparse MoE
Total parameters	119B
Activated parameters	~6B (~8B incl. embeddings/output)
Routed experts	128
Experts selected per token	4 (top-4)
Context length	256,000 tokens
Modalities	Text, image
Reasoning control	reasoning_effort: none / high
Licence	Apache 2.0
Minimum practical infrastructure	~4× H100-class GPUs

Mistral Medium 3.5, by contrast, is a straightforward 128B dense model: every parameter runs on every token, no expert routing, simpler to reason about operationally but proportionally more compute per token than Small 4's sparse design.

---

3. Architecture

3.1 Top-4-of-128 expert routing

Small 4 routes each token to 4 of 128 experts:

Top-4 sits in the middle of the sparsity spectrum covered across this section — more selective than DeepSeek V3's larger routed-expert counts, less selective than Kimi K3's top-16-of-896. Fewer experts per token generally means simpler routing and lower inter-GPU communication overhead in expert-parallel serving, at some cost to the per-token combination flexibility a larger top-k affords.

3.2 A binary reasoning_effort switch

Where Qwen 3.5, Gemma 4 and DeepSeek expose thinking/non-thinking as an on/off toggle, Mistral Small 4 exposes it as an explicit reasoning_effort request parameter with (at minimum) none and high settings:

Setting	Behaviour	Good for
none	Direct answer, no visible reasoning trace	Extraction, short factual queries, latency-sensitive chat
high	Extended internal reasoning before the final answer	Coding, multi-step maths, agentic planning

Because this is a request-level parameter rather than a separate model, a single Small 4 deployment can serve both latency-sensitive and accuracy-sensitive traffic from the same endpoint.

3.3 Native multimodality (Pixtral lineage)

Small 4's vision capability descends from Mistral's Pixtral line: images are patch-encoded and projected into the shared embedding space alongside text tokens, enabling document, chart, screenshot and photo understanding within the same unified checkpoint that also handles reasoning and coding — no separate vision-specialist deployment required.

3.4 Coding capability (Devstral lineage)

The Devstral lineage contributes agentic coding behaviour: repository navigation, multi-file editing, and tool-call formatting suitable for coding-agent harnesses, trained and evaluated the same way as the rest of the 2026 field's agentic coding RL (execution-verified rewards rather than only preference labels).

---

4. Training and post-training

Mistral's public technical materials (the original 7B paper, arXiv:2310.06825, and the Mixtral paper, arXiv:2401.04088) establish the lineage this family builds on: dense-then-sparse architectural progression, sliding-window attention experiments in earlier releases, and a consistent focus on parameter efficiency relative to peer model classes. Small 4's specific consolidation of reasoning, vision and coding implies a multi-objective post-training process — supervised fine-tuning and RL across reasoning traces, vision-grounded instructions and verified coding trajectories within one training run, rather than three separately fine-tuned specialist checkpoints merged after the fact.

Mistral publishes fine-tuning documentation (see references) enabling teams to further adapt Small 4 to a narrower domain, a capability particularly relevant given how many separate capabilities Small 4 already covers out of the box.

---

5. Deployment

5.1 Minimum infrastructure

Mistral documents a starting point of approximately 4× H100-class GPUs for Small 4 — a comparatively modest requirement next to trillion-parameter MoE flagships such as Kimi K3 or DeepSeek V4-Pro, and one reason Small 4 is positioned as a practical self-hosting choice for mid-sized organisations rather than only hyperscale operators.

Deployment goal	Suggested starting point
Development / evaluation	1–2 H100-class GPUs at 4-bit quantisation
Production, moderate concurrency	~4× H100-class GPUs at native or 8-bit precision
Mistral Medium 3.5 (128B dense)	Comparable or larger GPU count; no MoE routing overhead but full dense compute per token

5.2 Serving engines

Engine	Notes
vLLM	OpenAI-compatible; documented Mistral MoE support
SGLang	RadixAttention prefix caching; multimodal request handling
llama.cpp	GGUF-quantised local inference for smaller-footprint deployments
Transformers	Reference implementation and fine-tuning

5.3 Example: vLLM server

export HF_TOKEN="your-hugging-face-token"

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  mistralai/Mistral-Small-4 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.9 \
  --enable-auto-tool-choice \
  --tool-call-parser mistral

5.4 Calling the endpoint with reasoning effort

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4",
    messages=[
        {"role": "user", "content": "Design a rate limiter for a multi-tenant API."}
    ],
    extra_body={"reasoning_effort": "high"},
    max_tokens=4096,
)

print(response.choices[0].message.content)

5.5 Multimodal request example

import base64
from pathlib import Path
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

encoded = base64.b64encode(Path("diagram.png").read_bytes()).decode("utf-8")

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
                {"type": "text", "text": "Identify any single points of failure in this architecture diagram."},
            ],
        }
    ],
    extra_body={"reasoning_effort": "high"},
)

print(response.choices[0].message.content)

5.6 Mistral Medium 3.5 deployment

For the dense 128B alternative:

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  mistralai/Mistral-Medium-3.5 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.9

Dense models need more GPUs for the same parameter count than an equivalent-sized MoE model, since every parameter computes on every token rather than a sparse subset — expect Medium 3.5 to require proportionally more compute per request than Small 4 despite having a comparable total parameter count.

5.7 Choosing between Small 4 and Medium 3.5

Consideration	Mistral Small 4 (MoE)	Mistral Medium 3.5 (dense)
Operational complexity	Higher (expert routing, MoE-aware serving)	Lower (standard dense serving)
Compute per token	Lower (~6-8B active of 119B total)	Higher (full 128B active)
Minimum infrastructure	~4× H100-class	Comparable or larger GPU count
Best fit	Cost-sensitive production traffic at scale	Teams preferring simpler operational model, or without MoE-aware serving expertise

---

6. Common misconceptions

6.1 "One unified model must be worse at each task than three specialists"

Not necessarily. Consolidation into Small 4 requires multi-objective post-training across reasoning, vision and coding within one run, which carries real risk of capability trade-offs, but it also removes the overhead of routing requests between three separately hosted models and maintaining three separate deployment pipelines. Whether a unified or specialist approach wins depends on your workload mix — evaluate on your own tasks rather than assuming consolidation is automatically a downgrade.

6.2 "~6B active means Small 4 needs less GPU memory than a 119B dense model"

Only for compute, not for storage. The full 119B-parameter checkpoint must still be resident or shardable across the serving cluster; the ~6B (or ~8B including embeddings/output) figure describes per-token compute, not the memory footprint of the deployed model.

6.3 "reasoning_effort: high always improves output quality"

As with every reasoning-effort switch in this section, high trades latency and token cost for a visible reasoning trace, which helps most on genuinely multi-step problems (coding, planning, multi-step maths) and adds limited value — sometimes even introducing unnecessary complexity — on simple, direct queries.

6.4 "4× H100 is a hard universal minimum"

Mistral's stated starting point assumes a specific precision and concurrency target. Lower concurrency, more aggressive quantisation, or accepting reduced context length can shrink the practical minimum further for development and evaluation use; conversely, production concurrency at full 256K context may need more than four GPUs. Treat published minimums as a starting benchmark, not a fixed requirement.

---

7. Troubleshooting local deployment

7.1 Out-of-memory at startup on a 4-GPU node

Reduce --max-model-len from the full 256K ceiling, lower --gpu-memory-utilization slightly, and confirm --tensor-parallel-size matches the actual GPU count before assuming the hardware itself is undersized.

7.2 Vision requests failing while text-only requests succeed

Confirm the deployed checkpoint and serving-engine version both support the Pixtral-lineage vision path — a mismatched or outdated engine version is a common cause of vision-specific request failures that do not affect text-only traffic.

7.3 Tool calls not appearing in message.tool_calls

Verify --enable-auto-tool-choice and the correct --tool-call-parser value for Mistral's tool-call format are both set; test with a minimal single-tool request before integrating into a larger agent harness.

7.4 Inconsistent behaviour switching between reasoning_effort values

Confirm the parameter is being passed correctly through extra_body (or the serving engine's equivalent mechanism) rather than as a top-level field the OpenAI-compatible schema does not recognise — silently ignored parameters are a common source of "it doesn't seem to change anything" reports.

---

8. Licence

Both Mistral Small 4 and Mistral Medium 3.5 are released under Apache 2.0 — the same permissive baseline used by Qwen 3.5, Gemma 4 and GPT-OSS in this section, with no field-of-use restriction or usage-threshold clause. As with every open-weight model here, Apache 2.0 covers the published weights and code; it is not a guarantee about the licensing status of every upstream training input.

---

9. Engineer reading checklist

• Why Small 4 unifying Magistral/Pixtral/Devstral is an operational simplification, not just a capability merge
• Top-4-of-128 expert routing and where it sits on the sparsity spectrum vs Kimi K3 and DeepSeek
• The ~6B vs ~8B active-parameter accounting distinction (with/without embeddings and output projection)
• reasoning_effort: none vs high as a per-request control, not a separate model
• Minimum practical infrastructure (~4× H100-class) relative to trillion-parameter flagships
• Dense (Medium 3.5) vs MoE (Small 4) trade-offs for operational simplicity vs compute efficiency

---

References

• mistral.ai/news/mistral-small-4
• docs.mistral.ai — models reference
• Hugging Face: mistralai/Mistral-Small-4
• mistral-inference reference repository
• Mistral 7B paper — arXiv:2310.06825
• Mixtral paper — arXiv:2401.04088
• Mistral fine-tuning documentation

Key takeaways

• Mistral Small 4 replaces three separate specialist lines (Magistral, Pixtral, Devstral) with one Apache 2.0 MoE checkpoint.
• 119B total / ~6B active with top-4-of-128 routing makes Small 4 self-hostable on roughly four H100-class GPUs.
• reasoning_effort is a per-request parameter, not a separate model — one deployment serves both fast and deep-reasoning traffic.
• Mistral Medium 3.5 remains the simpler dense alternative when MoE operational complexity is not worth the compute savings.

What to do next

Compare with Gemma 4 for another efficiency-focused open-weight family, browse Open source models, or return to the AI model landscape for full category-wide selection guidance.