Command A+: Architecture, Training and Deployment

Command A+ is Cohere's Apache 2.0 open-weight Mixture-of-Experts model for enterprise agents, retrieval-augmented generation and sovereign or private deployment. It has roughly 218 billion total parameters with about 25 billion active per token, supports 128,000 tokens of input with up to 64,000 tokens of generation, native text, image and tool-calling, and 48 languages.

Executive summary

Command A+ is built for a specific buyer: enterprises that need agentic, RAG-heavy workloads under strict data-sovereignty or private-deployment requirements, not a general leaderboard-topping chatbot. Its minimum deployment footprint (roughly two H100-class GPUs at 4-bit W4A4 quantisation, or a single B200) is deliberately modest for its capability class, and Cohere pairs the model with speculative decoding and tokenizer-efficiency work aimed squarely at reducing enterprise inference cost.

When to use this page

• Selecting a model for enterprise RAG, tool-using agents, or sovereign/private-cloud deployment requirements.
• Sizing minimum GPU infrastructure for Command A+ (H100 vs B200 paths).
• Understanding the asymmetric 128K-input / 64K-generation context design.
• Pair with Mistral for another enterprise-oriented efficient MoE design, Open source models and the AI model landscape.

---

1. What kind of model is Command A+?

Open-weight (Apache 2.0), decoder-based, multimodal, tool-using Mixture-of-Experts model designed for enterprise agentic and RAG workloads, including sovereign and private deployment.

1.1 Apache 2.0, enterprise-first positioning

Command A+ ships under Apache 2.0 — the same permissive baseline as Qwen 3.5, Gemma 4, Mistral and GPT-OSS. Cohere's go-to-market and documentation emphasise enterprise agents, RAG pipelines and sovereign/private deployment more explicitly than most other open-weight releases in this section, which tend to lead with general capability or coding benchmarks first.

1.2 Asymmetric context: 128K input, 64K generation

Command A+'s context is split into two distinct limits rather than one shared budget:

Direction	Limit
Input (prompt, retrieved documents, tool definitions, history)	128,000 tokens
Generation (model output per turn)	64,000 tokens

This asymmetry reflects the model's target workloads directly: enterprise RAG and agentic tasks typically involve large retrieved context (documents, search results, tool schemas, conversation history) but comparatively bounded output (a synthesised answer, a structured tool call, a summarised report) — so the input budget is set roughly double the generation budget rather than treating both directions identically.

1.3 Multimodal and multilingual scope

Command A+ handles text, image and tool calls natively, and supports 48 languages — a scope aimed at multinational enterprises that need one deployment to serve varied language and document-type requirements rather than maintaining separate regional models.

---

2. Model configuration

Component	Command A+
Architecture	Sparse MoE
Total parameters	~218 billion
Activated parameters	~25 billion
Maximum input	128,000 tokens
Maximum generation	64,000 tokens
Modalities	Text, image
Tool use	Native tool/function calling
Languages	48
Licence	Apache 2.0
Minimum practical infrastructure	~2× H100-class GPUs (W4A4) or 1× B200

---

3. Architecture

3.1 218B total / ~25B active

Command A+'s active-parameter ratio (~11–12% of total) sits toward the denser end of the sparsity spectrum covered in this section compared with, for example, Kimi K3's ~104B-of-2.8T (~3.7%) or Mistral Small 4's ~6B-of-119B (~5%). A comparatively higher active fraction can mean more computation per token relative to total capacity, but also potentially simpler routing dynamics and more consistent per-token latency — a reasonable trade-off for enterprise workloads where predictable latency matters as much as raw efficiency.

3.2 Speculative decoding for enterprise cost control

Cohere documents speculative decoding support for Command A+, following the same draft-then-verify pattern used across the field: a smaller or lighter-weight draft mechanism proposes candidate tokens which the full model verifies in a single pass, improving throughput per GPU-hour without changing the final output distribution when verification succeeds. For enterprises billed on inference infrastructure cost, this directly targets total cost per completed task rather than only latency.

3.3 Tokenizer efficiency

Cohere reports tokenizer efficiency gains for Command A+ — a more efficient tokenizer reduces the number of tokens required to represent a given amount of text (particularly relevant across Command A+'s 48 supported languages), which lowers both cost and effective context consumption for the same underlying content, independent of any change to the model's architecture or weights.

3.4 Native tool calling for agents

Command A+ is trained with tool-calling as a first-class capability rather than an afterthought bolted on through prompt engineering, consistent with the enterprise-agent positioning: structured function-call generation, multi-turn tool-result incorporation, and RAG-style citation of retrieved sources are treated as core trained behaviours.

3.5 Comparison with other enterprise-oriented open-weight models

Model	Total/active params	Active fraction	Primary positioning
Command A+	218B / ~25B	~11–12%	Enterprise RAG, agents, sovereign deployment
Mistral Small 4	119B / ~6B	~5%	Unified reasoning/vision/coding checkpoint
GLM-5	Varies by checkpoint	—	Long-horizon coding agents

Command A+'s comparatively higher active-parameter fraction reflects its enterprise-latency-predictability positioning more than a fundamentally different architectural family — all three remain sparse MoE designs following the shared/routed expert pattern common across this section.

---

4. Training and post-training

Cohere's public materials for Command A+ (see references) emphasise training and evaluation specifically oriented around:

• RAG-grounded generation — answering from retrieved documents with accurate citation rather than relying purely on parametric knowledge.
• Multi-turn tool use — sustained agentic sessions involving repeated tool calls and result incorporation.
• Multilingual instruction-following — consistent behaviour across the 48 supported languages rather than English-first quality with weaker secondary-language support.
• Enterprise safety and refusal behaviour — tuned for business deployment contexts (compliance-sensitive refusals, appropriate escalation) as distinct from a general consumer-chat safety profile.

Cohere's related North Mini Code work (referenced in Cohere's blog) indicates continued investment in code-specific capability alongside the broader Command line, though Command A+ itself is positioned primarily as the general enterprise-agent and RAG flagship rather than a dedicated coding specialist.

---

5. Deployment

5.1 Minimum infrastructure

Cohere documents two practical minimum paths:

Path	Configuration
H100-class	~2× H100 GPUs using W4A4 quantisation (4-bit weights, 4-bit activations)
B200-class	1× NVIDIA B200 GPU

This is a notably modest footprint for a 218B-total MoE model, achieved through aggressive low-precision quantisation on the H100 path and the much larger per-GPU memory of a single B200 on the newer path — both meaningfully below the multi-node requirements of trillion-parameter flagships such as Kimi K3 or DeepSeek V4-Pro.

5.2 Sovereign and private deployment

Because Command A+ can run within a modest, well-defined hardware footprint, it is a realistic candidate for fully private or sovereign deployment — inside an enterprise's own data centre or a sovereign cloud region, with no requirement to send data to an external API — which is precisely the deployment pattern Cohere's enterprise positioning targets.

5.3 Serving engines

Engine	Notes
vLLM	OpenAI-compatible; documented Command A+ support including W4A4
SGLang	RadixAttention prefix caching; multimodal and tool-call support
cohere-toolkit	Cohere's own reference application stack for RAG and agent deployment on top of Command models

5.4 Example: vLLM server with W4A4 quantisation

export HF_TOKEN="your-hugging-face-token"

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  CohereLabs/command-a-plus-w4a4 \
  --trust-remote-code \
  --tensor-parallel-size 2 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.9 \
  --enable-auto-tool-choice \
  --tool-call-parser command-r

5.5 Calling the endpoint for a RAG request

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="CohereLabs/command-a-plus-w4a4",
    messages=[
        {
            "role": "system",
            "content": "Answer only from the provided documents and cite the source for each claim.",
        },
        {
            "role": "user",
            "content": "Document: [Q3 policy excerpt]\n\nQuestion: What is the refund window for enterprise customers?",
        },
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

5.6 Building a RAG/agent stack with cohere-toolkit

cohere-toolkit provides a reference implementation for retrieval, citation formatting, tool registration and conversation management on top of a Command A+ endpoint — a faster starting point than assembling an equivalent RAG/agent stack from a raw OpenAI-compatible client.

5.7 Single-GPU B200 deployment

docker run --rm \
  --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  CohereLabs/command-a-plus \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.9

A single B200's larger HBM capacity can host Command A+ without splitting across multiple GPUs, simplifying the deployment topology relative to the two-GPU H100 W4A4 path, at the cost of depending on newer-generation hardware availability.

5.8 Deployment path decision guide

Consideration	2× H100 (W4A4)	1× B200
Hardware availability	Broadly available today	Newer generation, more limited availability
Precision	4-bit weights and activations	Higher precision achievable on a single GPU
Topology complexity	Requires tensor parallelism across 2 GPUs	Single-GPU simplicity
Best fit	Organisations with existing H100 fleets	Organisations provisioning newer hardware

Both paths are explicitly documented minimums rather than the only viable configurations — production deployments with higher concurrency requirements should scale beyond either starting point based on measured load.

---

6. Common misconceptions

6.1 "128K input and 64K generation means 192K total context"

Treat them as two separate, purpose-specific budgets rather than a combined pool. Depending on the serving engine's exact accounting, input and output may or may not share the same underlying KV-cache allocation — confirm your specific engine's behaviour rather than assuming a simple additive relationship.

6.2 "Sovereign deployment means no cloud infrastructure at all"

Sovereign deployment typically means infrastructure within a specific legal jurisdiction or under an organisation's exclusive control — this can still be cloud infrastructure (a sovereign cloud region, a dedicated private cloud) rather than strictly on-premises hardware. The requirement is about data control and jurisdiction, not necessarily about avoiding cloud infrastructure entirely.

6.3 "25B active parameters means Command A+ is 'smaller' than it looks"

Its ~25B active parameters are proportionally higher relative to its ~218B total than several other MoE models in this section (compare Kimi K3's ~104B of 2.8T). This denser activation ratio means more compute per token than a highly sparse model of similar total size — plan compute budgets accordingly rather than assuming a low sparsity ratio automatically implies low resource needs.

6.4 "Native tool calling means no validation is needed"

Trained tool-calling behaviour improves the reliability of correctly formatted function calls, but does not eliminate the need for application-level validation, authorisation and sandboxing before executing any tool call — the same requirement applies to every tool-calling model in this section, regardless of how well-trained the tool-calling behaviour is.

---

7. Troubleshooting local deployment

7.1 Out-of-memory on the documented 2× H100 W4A4 configuration

Confirm the W4A4-quantised checkpoint variant is actually being loaded (not a higher-precision original), and reduce --max-model-len from the full 128K input ceiling for initial testing before scaling toward production concurrency.

7.2 RAG citations missing or inaccurate

Verify the system prompt explicitly instructs citation behaviour and that retrieved documents are clearly delimited in the prompt structure — citation quality is heavily influenced by prompt structure and retrieval quality, not solely by the model's trained citation capability.

7.3 Inconsistent behaviour across the 48 supported languages

If quality varies significantly by language, evaluate the specific language pair or language against Cohere's documented evaluation results rather than assuming uniform quality across all 48 — multilingual capability commonly varies by language even within a single well-trained multilingual model.

7.4 Speculative decoding not improving throughput

Confirm speculative decoding is actually enabled in the serving configuration and that the draft-acceptance rate is being monitored — a low acceptance rate for your specific workload can make speculative decoding provide little or even negative benefit, the same caveat that applies to every speculative-decoding implementation in this section.

---

8. Licence

Command A+ is released under Apache 2.0 — the same permissive baseline as Qwen 3.5, Gemma 4, Mistral and GPT-OSS. As with every model in this section, Apache 2.0 covers the published weights and code; it does not, by itself, describe Cohere's training-data sourcing or provide a guarantee about upstream data provenance — run standard legal review before high-stakes commercial or sovereign deployment.

---

9. Engineer reading checklist

• Why 128K input / 64K generation is an asymmetric, RAG-oriented context design rather than a single shared budget
• 218B total / ~25B active as a comparatively higher-density MoE point vs Kimi K3 or Mistral Small 4
• Speculative decoding and tokenizer efficiency as enterprise cost-control levers, distinct from architecture changes
• Why a modest H100/B200 footprint makes sovereign/private deployment realistic
• Native tool calling and RAG citation as first-class trained behaviours, not prompt-engineered add-ons
• cohere-toolkit as a reference RAG/agent stack rather than building one from scratch

---

References

• cohere.com/blog/command-a-plus
• docs.cohere.com/docs/models
• Hugging Face: CohereLabs/command-a-plus
• cohere-toolkit repository
• Cohere blog: North Mini Code

Key takeaways

• Command A+ targets enterprise RAG, agents and sovereign deployment specifically, with an asymmetric 128K-input/64K-generation context to match.
• Its ~218B/~25B MoE configuration is denser (higher active fraction) than many peers, trading some efficiency for latency predictability.
• Roughly 2× H100 (W4A4) or a single B200 is a genuinely modest footprint for private or sovereign deployment.
• cohere-toolkit accelerates building a production RAG/agent stack on top of the raw model endpoint.

What to do next

Compare with Mistral for another enterprise-efficient MoE design, browse Open source models, or return to the AI model landscape for enterprise RAG and agent selection guidance.