Skip to main content

Qwen 3.5: Architecture, Training and Deployment

Qwen 3.5 is Alibaba's flagship open-weight family for mid-2026: a native multimodal, hybrid-attention Mixture-of-Experts line released under the Apache 2.0 licence. The largest documented open configuration, 397B-A17B, has roughly 397 billion total parameters with about 17 billion active per token; a smaller 122B-A10B variant targets teams that cannot operate frontier-scale clusters. Both support thinking and non-thinking modes and context lengths pushing toward 1,048,576 tokens.

1. What kind of model is Qwen 3.5?​

Qwen 3.5 can be summarised as:

Open-weight, decoder-based, natively multimodal, hybrid linear-attention Mixture-of-Experts family with switchable reasoning depth.

1.1 Open-weight, Apache 2.0​

Alibaba publishes Qwen 3.5 checkpoints under Apache 2.0 — a permissive, OSI-approved software licence with no field-of-use restriction and no attribution clause beyond standard notice preservation. This is more commercially friendly than Kimi K3's custom licence or Llama 4's Community Licence.

Apache 2.0 covers the weights and inference code. It does not, by itself, make Qwen 3.5 "open source" in the Open Source Initiative's model-specific sense — Alibaba has not published the full pre-training corpus, exact data-selection pipeline, or every intermediate checkpoint. Treat Qwen 3.5 as open-weight with a highly permissive licence, not as a fully reproducible open-source research release.

1.2 A family, not one model​

VariantTotal paramsActive paramsNotes
Qwen3.5-397B-A17B~397B~17BFlagship; long context toward 1M
Qwen3.5-122B-A10B~122B~10BSmaller MoE; more practical self-hosting footprint

Both share the same architectural family (Gated DeltaNet + attention + MoE) and post-training recipe; they differ in depth, expert count and total capacity.

1.3 Native multimodality​

Qwen 3.5 fuses text and vision early in the network rather than bolting a caption model onto a frozen language backbone. Images are patch-encoded and projected into the shared embedding space alongside text tokens before entering the hybrid backbone, so the model can reason jointly over a screenshot, a diagram and the surrounding text in one pass.

1.4 Thinking and non-thinking modes​

Qwen 3.5 exposes an explicit switch between:

  • Non-thinking — direct answer generation, lower latency, appropriate for extraction, formatting, short factual queries.
  • Thinking — the model emits an internal reasoning trace before the final answer, appropriate for multi-step coding, maths and planning tasks.

Unlike a single fixed "reasoning model", the same checkpoint serves both a chat-latency budget and a deep-reasoning budget, selected per request.

1.5 Multilingual scope​

The Qwen3 family narrative describes support across roughly 201 languages and dialects, backed by an expanded vocabulary (Qwen3.5 grew the tokenizer to approximately 250,000 tokens). A larger, more carefully balanced vocabulary reduces the number of tokens needed to represent non-English text, which lowers both cost and latency for multilingual workloads.

2. Model configuration​

ComponentQwen3.5-397B-A17B (documented)
ArchitectureHybrid Gated DeltaNet + attention, sparse MoE
Total parameters~397 billion
Activated parameters~17 billion
Smaller sibling122B-A10B
Context lengthExtends toward 1,048,576 tokens
ModalitiesText, image (native fusion)
Reasoning modesThinking / non-thinking
Vocabulary~250,000 tokens (family-wide)
Multilingual coverage~201 languages/dialects (family narrative)
LicenceApache 2.0

As with any MoE model, total parameters describe checkpoint size and storage; active parameters describe the compute performed per generated token. A 397B/17B model is far cheaper to run than a dense 397B model, but the full 397B must still be resident (or shardable) across the serving cluster.

3. Architecture​

3.1 Dense vs MoE, one more time​

A dense transformer runs every parameter on every token. An MoE transformer routes each token to a small subset of specialist feed-forward "experts", plus usually one or more shared experts that process every token unconditionally:

Shared experts capture broadly useful transformations (general syntax, common world knowledge); routed experts specialise statistically during training. Qwen 3.5 uses this shared/routed split, consistent with the wider 2026 MoE convention established by DeepSeek and adopted across most frontier open-weight releases.

3.2 Gated DeltaNet​

Gated DeltaNet is a linear-attention-family recurrent layer. Instead of comparing a token against every previous token (the quadratic cost of full self-attention), it maintains a fixed-size recurrent state updated with a delta rule — the state is corrected toward new information rather than simply appended to:

S_t = decay(S_{t-1}) + delta(k_t, v_t, q_t)

A learned gate additionally controls how much of the existing state is retained versus overwritten per channel, similar in spirit to Kimi's Gated MLA output gate and to Mamba's selective-state mechanism, but implemented as a delta-rule linear-attention variant rather than a state-space model. The practical effect: near-linear cost in sequence length for the Gated DeltaNet layers, at the expense of the same limitation every recurrent/linear-attention mechanism has — a compressed state cannot perfectly recall every exact earlier token relationship.

3.3 Why keep full attention at all?​

Qwen 3.5 interleaves Gated DeltaNet layers with a smaller number of conventional full-attention layers, following the now-common hybrid pattern (see also Kimi K3's KDA + Gated MLA, and Nemotron 3's Mamba-2 + attention):

Layer typeRole
Gated DeltaNetEfficient long-sequence state carrying; approximately linear cost
Full attentionPeriodic exact, global token-to-token retrieval
MoE feed-forwardSpecialised nonlinear per-token transformation

Pure linear-attention stacks tend to underperform on tasks requiring precise long-range copy or exact retrieval (e.g. "what was in cell B14 of the table 80K tokens ago"). Interleaving a smaller number of full-attention layers recovers most of that precision while keeping the majority of the network's sequence-processing cost close to linear.

3.4 Active vs total parameters, applied to Qwen 3.5​

For 397B-A17B:

  • 397B total — all routed and shared expert weights, attention projections, embeddings.
  • 17B active — the shared experts plus the top-k routed experts actually invoked for a given token, plus the always-active attention/DeltaNet/embedding layers.

The proportion of experts activated (~4–5% of total capacity) is broadly consistent with the industry direction toward higher sparsity — more total experts, fewer selected per token — that also shows up in Kimi K3 (16 of 896) and DeepSeek V4.

3.5 Long-context strategy​

Reaching toward one million tokens with a hybrid design relies on the Gated DeltaNet layers carrying the bulk of sequence length at near-linear cost, while full-attention layers are the primary consumer of KV-cache memory. As with every long-context model, serving frameworks must manage two distinct memory pools — the fixed-size recurrent state for DeltaNet layers, and the growing KV cache for full-attention layers — rather than a single uniform cache.

4. Training and post-training​

4.1 Pre-training data mixture​

Qwen's published technical reports describe a pre-training mixture spanning web text, code, mathematics, multilingual corpora and synthetic data generated by earlier Qwen models, combined with quality classifiers and deduplication — broadly the same pipeline shape used across the 2026 frontier field (see the landscape guide for the generic pipeline).

4.2 Reasoning-budget training​

Training thinking and non-thinking behaviour into a single checkpoint requires explicit supervision at both ends of the reasoning-effort spectrum: short, direct trajectories for non-thinking behaviour, and longer verified chain-of-thought trajectories for thinking mode, so the model does not default to one style regardless of the request.

4.3 Agentic coding reinforcement learning​

Qwen 3.5 and the related Qwen3-Coder line are trained with reinforcement learning against verifiable coding signals: passing unit tests, successful compilation, correct tool-call syntax, and multi-step repository edits validated by execution rather than only by human preference. This mirrors the broader shift described in the landscape guide toward RL rewarded on task completion rather than surface-level answer quality.

4.4 Supervised fine-tuning and distillation​

As with other large MoE families, a smaller number of expensive, high-quality trajectories (agentic, multi-turn, tool-using) are used for supervised fine-tuning, with distillation from larger or more heavily RL-trained internal checkpoints used to transfer capability into the smaller 122B-A10B variant without repeating the full RL budget.

5. Deployment​

5.1 Can it run on a single workstation?​

No, not the 397B-A17B flagship. Even with only ~17B active parameters computed per token, the full 397B-parameter checkpoint must be available to the serving engine (sharded across GPUs or offloaded), and MoE routing means different tokens in the same batch may touch different experts — so partial loading is not a simple substitute for having the full model addressable.

The 122B-A10B variant is meaningfully more approachable: a single high-memory server (for example 4–8 modern datacentre GPUs, or fewer at aggressive quantisation) can host it, whereas 397B-A17B realistically needs a multi-GPU node at minimum, and a multi-node cluster for production-grade concurrency and full context length.

5.2 Serving engines​

Qwen 3.5 is supported by the same engines used across the open-weight ecosystem:

EngineNotes
vLLMOpenAI-compatible server; PagedAttention KV cache; broad MoE and hybrid-layer support
SGLangRadixAttention prefix caching; strong multimodal and structured-output support
KTransformersCPU/GPU hybrid inference aimed at running very large MoE checkpoints with limited GPU memory by offloading routed experts to CPU RAM

KTransformers is particularly relevant for Qwen 3.5's largest configuration: because only a small fraction of experts are active per token, offloading the (mostly idle) remainder to CPU memory and streaming only the selected experts to the GPU can make 397B-class MoE inference viable on far smaller GPU budgets than a dense model of the same total size, at a decode-speed cost.

5.3 Example: vLLM OpenAI-compatible server​

export HF_TOKEN="your-hugging-face-token"

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  Qwen/Qwen3.5-122B-A10B \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --tensor-parallel-size 4 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3

5.4 Calling the local endpoint​

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-122B-A10B",
    messages=[
        {"role": "user", "content": "Refactor this function for O(n) complexity."}
    ],
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)

print(response.choices[0].message.content)

The enable_thinking flag (naming may vary by serving-engine version) toggles thinking mode at the template level; always confirm the current flag name against the model card before deploying, since chat-template controls change between releases.

5.5 Multi-node deployment for 397B-A17B​

Production-grade concurrency at the flagship size typically needs more than one node:

export HEAD_IP="10.0.0.10"
export IFACE_NAME="ib0"
export HF_TOKEN="your-hugging-face-token"

docker run --rm \
  --gpus all \
  --ipc=host \
  --network=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN="$HF_TOKEN" \
  -e NCCL_SOCKET_IFNAME="$IFACE_NAME" \
  -e GLOO_SOCKET_IFNAME="$IFACE_NAME" \
  vllm/vllm-openai:latest \
  Qwen/Qwen3.5-397B-A17B \
  --trust-remote-code \
  --tensor-parallel-size 16 \
  --pipeline-parallel-size 2 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.9

As with every multi-node deployment in this section, cross-node all-to-all traffic (for MoE expert dispatch) needs a high-bandwidth interconnect — plain Ethernet is unlikely to sustain production throughput.

5.6 Comparison with sibling hybrid-attention models​

ModelRecurrent/linear mechanismFull-attention layersPositional scheme
Qwen 3.5Gated DeltaNetPeriodicStandard (RoPE-family)
Kimi K3Kimi Delta Attention (KDA)Gated MLA, periodicNoPE
Nemotron 3 SuperMamba-2PeriodicStandard

All three converge on the same broad idea — recurrent/linear layers carry the bulk of sequence length, periodic full attention recovers exact global retrieval — while differing in the specific recurrent formulation and positional strategy.

5.7 Context-length planning​

Follow the same discipline recommended for every long-context hybrid model: do not default to the maximum context. Start a new deployment at 32K–128K, benchmark time-to-first-token and concurrency, and only raise the limit toward the 1M ceiling once the workload demonstrably needs it and the recurrent-state and KV-cache pools have been sized accordingly.

6. Common misconceptions​

6.1 "17B active means it runs like a 17B model"​

Not for storage or serving-infrastructure purposes. The router can send different tokens in the same batch to different experts, so the serving engine must have all 397B parameters addressable (in GPU memory, or offloaded and streamed) even though any single token only touches ~17B of them. Latency and throughput benefit from the smaller active-compute figure; memory planning does not.

6.2 "Thinking mode always gives a better answer"​

Thinking mode spends more tokens and more latency on a visible reasoning trace before the final answer. For simple extraction, classification or short factual queries it commonly adds cost without improving correctness, and can occasionally introduce new errors by over-elaborating a task that had an obvious direct answer. Reserve it for genuinely multi-step problems.

6.3 "Gated DeltaNet is the same as Mamba"​

They are both linear-cost recurrent alternatives to full attention, but Gated DeltaNet is a delta-rule linear-attention variant (it maintains a state expressed in the same query/key/value formalism as attention), while Mamba/Mamba-2 is a selective state-space model with a different mathematical parameterisation. Serving frameworks sometimes group both under generic "hybrid model" tooling, but their internal update rules and recommended serving configuration are not interchangeable.

6.4 "397B-A17B is always the right choice over 122B-A10B"​

Only if the workload and infrastructure justify it. The larger model is more expensive to host, harder to fit on a single node, and slower to iterate on for fine-tuning experiments. Many production coding-agent workloads are well served by 122B-A10B, reserving 397B-A17B for cases where evaluation shows a measurable, business-relevant capability gap.

7. Troubleshooting local deployment​

7.1 Out-of-memory at startup​

Likely causes: --gpu-memory-utilization set too high for the available cluster; full-context KV-cache pre-allocation before confirming actual concurrency needs; attempting 397B-A17B on a node sized for 122B-A10B.

Responses: reduce --max-model-len, lower memory utilisation slightly, confirm tensor-parallel size matches the actual GPU count, or move to the smaller variant.

7.2 Low throughput despite low active-parameter count​

MoE throughput depends heavily on routing efficiency and all-to-all communication between GPUs holding different experts, not only on the active-parameter figure. Check expert-parallel configuration, confirm the interconnect (NVLink/InfiniBand) is actually being used for expert dispatch, and profile whether load is balanced across experts rather than concentrated on a popular subset.

7.3 Thinking-mode output appearing in the final answer​

Confirm the serving engine's reasoning parser is enabled and matches the model's exact template version — chat-template and parser mismatches are a common source of reasoning content leaking into (or being stripped from) the user-facing response after a model or engine upgrade.

7.4 Tool calls rendered as plain text​

Ensure --enable-auto-tool-choice and the correct --tool-call-parser value for the deployed Qwen version are set; verify with a minimal tool-calling request before integrating into a full agent harness.

8. Licence​

Qwen 3.5 weights are released under Apache 2.0 — no field-of-use restriction, no revenue threshold, no separate commercial licence required. This is more permissive than Kimi K3's custom licence and Llama 4's Community Licence, and comparable to Mistral's and GPT-OSS's Apache 2.0 releases. As always: Apache 2.0 covers the weights and code Alibaba chose to publish, not an implicit guarantee that every upstream training input was itself unencumbered — run standard legal review before high-stakes commercial redistribution.

9. Engineer reading checklist​

  • Dense vs MoE, and why active parameters ≠ total parameters
  • Gated DeltaNet: delta-rule state update and per-channel gating
  • Why hybrid models interleave linear attention with full attention
  • Shared experts vs routed experts and top-k routing
  • Thinking vs non-thinking mode and when to use each
  • Agentic coding RL rewarded on execution, not just preference
  • KTransformers-style CPU expert offload for large MoE checkpoints
  • Recurrent-state pool vs KV-cache pool sizing for hybrid serving

References​

  • Hugging Face: Qwen/Qwen3.5-397B-A17B
  • Hugging Face: Qwen/Qwen3.5-122B-A10B
  • GitHub: github.com/QwenLM/Qwen3
  • Qwen3 Technical Report — arXiv:2505.09388
  • Qwen3-Coder model card and documentation
  • qwenlm.github.io blog posts
  • qwen-code agentic coding harness repository
Last updated on

Discussion

Comments​

Share feedback or questions about this page. No account required.

Loading comments…

Occasional notes on AI delivery.

Unsubscribe anytimePrivacy