gpt-oss: Architecture, Training and Deployment

gpt-oss is OpenAI's Apache 2.0 open-weight reasoning model family, released as two sizes: gpt-oss-120b (~117B total / ~5.1B active) and gpt-oss-20b (~21B total / ~3.6B active). Both use alternating dense and locally banded sparse attention, grouped-query attention, rotary positional embeddings, native MXFP4 quantised weights, and communicate through OpenAI's structured Harmony response format with configurable reasoning effort.

Executive summary

gpt-oss is OpenAI's first mainline open-weight release in years, and its design choices read as a direct port of frontier-model serving discipline into a self-hostable package: MXFP4 quantisation gets gpt-oss-120b to roughly 80GB and gpt-oss-20b to roughly 16GB — single high-end GPU or workstation territory — while the Harmony format and configurable reasoning effort mirror the request-level controls OpenAI's hosted API already exposes. Apache 2.0 licensing removes the usual open-weight commercial friction entirely.

When to use this page

• Selecting a self-hostable reasoning model that fits on a single high-end GPU (120b) or a single consumer/workstation GPU (20b).
• Understanding the Harmony response format and why it differs from a plain OpenAI-style chat message.
• Sizing memory for MXFP4-native weights vs post-hoc quantisation of a higher-precision checkpoint.
• Pair with Open source models, the AI model landscape and Kimi K3 for a contrasting quantisation-aware training approach.

---

1. What kind of model is gpt-oss?

Open-weight (Apache 2.0), decoder-based, sparse MoE reasoning model family, natively quantised to MXFP4 and served through a structured response format.

1.1 Apache 2.0 — OpenAI's most permissive open release

gpt-oss ships under Apache 2.0, the same permissive licence family as Qwen 3.5, Gemma 4, Mistral and GLM-5. This is notable specifically because it comes from OpenAI, whose flagship models (the GPT-5.6 family) are entirely closed — gpt-oss is explicitly the "here is a genuinely open-weight tier below the frontier API" release rather than a partial preview of the closed flagship's architecture.

1.2 Two sizes, both MoE

Variant	Total params	Active params	Fits within (native quant)
gpt-oss-120b	~117B	~5.1B	~80GB
gpt-oss-20b	~21B	~3.6B	~16GB

Both activate a small fraction of total parameters per token (roughly 4–17%), consistent with the field-wide MoE sparsity trend, and both are documented to fit within specific memory budgets under their native MXFP4 quantisation — a directly actionable planning number rather than a rough estimate requiring separate quantisation research.

1.3 Context and reasoning

Both sizes support up to 128,000 tokens of context and expose configurable reasoning effort, following the same reasoning-budget pattern used across the field (compare Qwen 3.5's thinking/non-thinking switch and Mistral Small 4's reasoning_effort parameter).

---

2. Model configuration

Component	gpt-oss-120b	gpt-oss-20b
Architecture	Sparse MoE, alternating dense/banded-sparse attention	Sparse MoE, alternating dense/banded-sparse attention
Total parameters	~117 billion	~21 billion
Activated parameters	~5.1 billion	~3.6 billion
Attention	Alternating dense and locally banded sparse, GQA, RoPE	Alternating dense and locally banded sparse, GQA, RoPE
Maximum context	128,000 tokens	128,000 tokens
Weight quantisation	MXFP4 (native)	MXFP4 (native)
Reasoning	Configurable effort	Configurable effort
Response format	Harmony	Harmony
Memory footprint (native quant)	~80GB	~16GB
Licence	Apache 2.0	Apache 2.0

---

3. Architecture

3.1 Alternating dense and locally banded sparse attention

gpt-oss layers alternate between two attention patterns rather than using one uniformly:

Locally banded sparse attention restricts each token's attention computation to a band of nearby positions (similar in spirit to Gemma 4's sliding-window local layers), reducing cost for the majority of layers while periodic dense layers preserve some full-sequence retrieval capacity — another variation on the local/global or sparse/dense hybrid pattern that recurs throughout this section, implemented here with fixed attention patterns rather than a recurrent state (KDA, Gated DeltaNet, Mamba-2) or a learned indexer (GLM-5's sparse attention).

3.2 Grouped-query attention (GQA)

GQA reduces the number of distinct key/value projections relative to the number of query heads — multiple query heads share a smaller set of key/value heads rather than each query head having its own dedicated KV projection:

This directly shrinks KV-cache memory relative to standard multi-head attention, at a small cost to per-head representational independence — a well-established efficiency technique used across much of the current open-weight field, applied here specifically to gpt-oss's dense attention layers.

3.3 Rotary positional embeddings (RoPE)

gpt-oss uses RoPE to encode token position — rotating query and key vectors by an angle proportional to their sequence position so that relative position information is naturally reflected in the attention dot product. This is the conventional counterpart to the position-free approaches used elsewhere in this section (Kimi K3's NoPE relies on KDA's recurrence for positional information instead).

3.4 MXFP4: native, not retrofitted

gpt-oss weights are published natively as MXFP4 — a 4-bit microscaling floating-point format where small groups of values share scaling information to preserve useful numerical range at low bit-width (the same format family Kimi K3 applies to its routed expert weights during quantisation-aware post-training). Because gpt-oss ships MXFP4 as the native published format, the documented memory footprints (~80GB for 120b, ~16GB for 20b) are directly achievable without a separate community re-quantisation step, unlike models where only a higher-precision original is published and the community must produce and validate its own 4-bit conversions.

3.5 The Harmony response format

Rather than a conventional flat role/content chat message, gpt-oss communicates through OpenAI's Harmony format — a structured response schema that explicitly separates channels such as reasoning, tool calls, and final user-facing output, giving a serving stack a well-defined way to distinguish "the model's internal reasoning trace" from "the message actually meant for the end user" without relying on ad hoc string-parsing heuristics.

Serving frameworks that support gpt-oss (vLLM, SGLang, transformers) implement a Harmony parser to split these channels correctly; using a generic chat-completion client without Harmony awareness risks leaking reasoning-channel content into the user-facing response, or misparsing structured tool calls as plain text.

---

4. Training and post-training

OpenAI's gpt-oss technical report (arXiv:2508.10925) documents the family's training approach:

• Pretraining on a broad web, code and reasoning-oriented corpus at MoE scale.
• Post-training with configurable reasoning effort, trained explicitly so that a single checkpoint can operate at different reasoning depths on request rather than requiring a separate model per effort level.
• Structured tool-calling training aligned with the Harmony format's dedicated tool-call channel, rather than reasoning about tool syntax purely through few-shot prompting.
• Safety-oriented post-training consistent with OpenAI's broader model-release practices, adapted for a self-hosted deployment context where the provider does not control the final serving environment.

As with every model in this section, OpenAI has not published the full pretraining dataset or complete training code alongside the weights — gpt-oss is open-weight, with a detailed technical report, not a fully reproducible open-source training pipeline.

---

5. Deployment

5.1 Hardware planning

Model	Native-quant footprint	Realistic hardware
gpt-oss-20b	~16GB	Single consumer/workstation GPU (e.g. a 24GB-class card)
gpt-oss-120b	~80GB	Single high-end datacentre GPU (e.g. 80GB-class), or split across two smaller GPUs

These figures are for the native MXFP4 weights specifically — do not assume the same footprint applies if reverting to a higher-precision checkpoint for fine-tuning, which will require substantially more memory during training even if inference later returns to MXFP4.

5.2 Serving engines

Engine	Notes
vLLM	Documented gpt-oss support including native MXFP4 and Harmony parsing
SGLang	RadixAttention prefix caching; Harmony-aware response handling
Transformers	Reference implementation, fine-tuning, research use

5.3 Example: vLLM server for gpt-oss-20b

export HF_TOKEN="your-hugging-face-token"

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  openai/gpt-oss-20b \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --max-model-len 65536 \
  --enable-auto-tool-choice \
  --tool-call-parser openai \
  --reasoning-parser gpt-oss

5.4 Calling the endpoint with reasoning effort

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    reasoning_effort="high",
    messages=[
        {"role": "user", "content": "Debug why this async function occasionally deadlocks."}
    ],
    max_tokens=2048,
)

message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Because the underlying wire format is Harmony, always confirm the serving engine version correctly separates the reasoning/analysis channel from the final channel — printing raw, unparsed model output risks exposing internal reasoning text that was never meant to be user-facing.

5.5 gpt-oss-120b on a single 80GB-class GPU

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  openai/gpt-oss-120b \
  --trust-remote-code \
  --gpu-memory-utilization 0.92 \
  --max-model-len 32768

Start with a conservative context length (well below the 128K ceiling) and raise it only after confirming KV-cache headroom on the target GPU, following the same discipline recommended for every model in this section.

5.6 gpt-oss-120b split across two smaller GPUs

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  openai/gpt-oss-120b \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.9

Splitting across two smaller GPUs (rather than one 80GB-class GPU) trades some inter-GPU communication overhead for broader hardware compatibility — useful when a single 80GB-class card is not available but two smaller GPUs are.

5.7 Choosing between gpt-oss-20b and gpt-oss-120b

Consideration	gpt-oss-20b	gpt-oss-120b
Native-quant footprint	~16GB	~80GB
Realistic hardware	Single consumer/workstation GPU	Single high-end datacentre GPU, or split across two
Best fit	Local development, cost-sensitive production, edge-adjacent deployment	Higher-capability self-hosted reasoning where a single-GPU footprint is still desired

Both sizes are explicitly designed to be self-hostable without a multi-node cluster — the choice is primarily about available hardware and required capability, not about crossing an infrastructure-complexity threshold the way trillion-parameter flagships in this section require.

---

6. Common misconceptions

6.1 "gpt-oss is a smaller version of GPT-5.6"

There is no public confirmation that gpt-oss shares architecture, training data or weights with OpenAI's closed GPT-5.6 family. Treat gpt-oss as its own documented open-weight release (with its own technical report, arXiv:2508.10925) rather than assuming it is a distilled or truncated version of the closed flagship.

6.2 "MXFP4 native publication means no accuracy trade-off exists"

Native MXFP4 pretraining/publication avoids the specific accuracy loss that comes from retrofitting quantisation onto weights never trained with that constraint, but 4-bit representation is still lower precision than 16-bit — there remains an inherent trade-off relative to a hypothetical higher-precision version of the same model, it is simply a trade-off OpenAI validated and accepted during development rather than one left to downstream users to discover.

6.3 "Harmony is just a different name for standard OpenAI chat messages"

Harmony explicitly separates reasoning/analysis, tool-call, and final-answer channels at the protocol level, which a plain role/content chat message format does not do. Treating Harmony output as an ordinary chat completion string without channel-aware parsing risks leaking internal reasoning content into user-facing output.

6.4 "20b and 120b differ only in size, with identical behaviour otherwise"

Both share the same architectural family (alternating dense/sparse attention, GQA, RoPE, MXFP4, Harmony), but different total training compute and parameter budgets generally produce different capability profiles beyond simple scale — validate the smaller model specifically for your task rather than assuming it is simply "120b but weaker in every dimension by a fixed ratio."

---

7. Troubleshooting local deployment

7.1 Out-of-memory despite the documented ~80GB / ~16GB figures

Confirm the native MXFP4 checkpoint variant is actually being loaded, and account separately for KV-cache memory at your target context length and concurrency — the documented footprints describe weight memory only, not the full runtime memory budget.

7.2 Reasoning content leaking into user-facing responses

This is the most common gpt-oss-specific issue: confirm the serving engine's Harmony parser is enabled and up to date, and inspect the raw response structure directly if reasoning text appears where only the final answer was expected.

7.3 Tool calls not recognised by the application

Verify the Harmony parser's tool-call channel is being correctly mapped to the OpenAI-compatible tool_calls field by the serving engine — a parser version mismatch after an engine upgrade is the most common cause of tool calls silently degrading to plain text.

7.4 Fine-tuning runs failing due to memory despite small native footprint

Fine-tuning typically requires higher-precision gradient and optimiser-state memory well beyond the native MXFP4 inference footprint — plan fine-tuning infrastructure independently from inference infrastructure, especially for gpt-oss-120b.

---

8. Licence

gpt-oss is released under Apache 2.0 — no field-of-use restriction, no usage-threshold clause, and no attribution requirement beyond standard notice preservation. This places it alongside Qwen 3.5, Gemma 4, Mistral, GLM-5 and Command A+ as one of the more commercially frictionless open-weight releases in this section, notably more permissive than Llama 4's custom Community Licence and Kimi K3's custom licence.

---

9. Engineer reading checklist

• Alternating dense / locally banded sparse attention as a fixed-pattern alternative to recurrent or learned-indexer long-context mechanisms
• Grouped-query attention and its direct KV-cache memory reduction
• Why native MXFP4 publication (not retrofitted quantisation) makes the ~80GB/~16GB figures directly actionable
• The Harmony format's channel separation (reasoning/analysis, tool calls, final) and why a Harmony-aware parser is required
• Configurable reasoning effort as a request-level parameter on a single checkpoint
• Why fine-tuning memory requirements can exceed native-quant inference figures

---

References

• openai.com/index/introducing-gpt-oss
• GitHub: github.com/openai/gpt-oss
• gpt-oss Technical Report — arXiv:2508.10925
• Hugging Face: openai/gpt-oss-120b and openai/gpt-oss-20b
• openai.com/open-models

Key takeaways

• gpt-oss-120b (~117B/5.1B active, ~80GB) and gpt-oss-20b (~21B/3.6B active, ~16GB) are natively MXFP4, so documented memory figures are directly achievable without community re-quantisation.
• Alternating dense/locally-banded-sparse attention with GQA and RoPE is a fixed-pattern efficiency approach, distinct from the recurrent and learned-indexer methods used elsewhere in this section.
• The Harmony response format requires a Harmony-aware parser — do not treat gpt-oss output as a plain chat-completion string.
• Apache 2.0 licensing makes gpt-oss one of the most commercially frictionless choices in this guide, notably from a vendor whose flagship models remain fully closed.

What to do next

Compare with Mistral and Qwen 3.5 for other Apache 2.0 reasoning-capable alternatives, browse Open source models, or return to the AI model landscape for full category-wide selection guidance.