37 posts tagged with "AI Engineering"

Knowledge distillation is a model-compression and capability-transfer technique in which a powerful teacher model provides training signals for a smaller student model.

Instead of requiring the student to rediscover every useful behaviour from raw internet-scale pretraining data, the student learns from the teacher’s outputs, probability distributions, internal representations, reasoning demonstrations or preferences.

A distilled model can therefore become:

• Faster at inference
• Less expensive to operate
• Smaller in memory
• Easier to deploy on limited hardware
• More specialised for a particular task
• More consistent than the original general-purpose model for a narrow workflow

However, an important distinction must be made:

The exact internal process used to train current proprietary Claude models is not publicly disclosed in full.

Anthropic publishes model reports, safety research and selected training information, but its public materials do not expose Claude’s weights, token logits, hidden states, training datasets or the complete teacher–student training recipe. Therefore, when people discuss “distilling Claude,” they may be referring to two different things:

1. Authorised distillation within an official platform, such as Amazon Bedrock’s documented Claude Sonnet-to-Haiku distillation workflow.
2. Black-box behavioural distillation, where permitted Claude outputs are collected and used to fine-tune another model.

Anthropic and AWS have publicly described an authorised workflow in which Claude 3.5 Sonnet generates synthetic training data, Claude 3 Haiku is trained and evaluated using that data, and the resulting distilled model is hosted for inference. This is primarily output-based behavioural distillation rather than traditional access to Claude’s internal logits or hidden layers.

Fifteen model families, three licensing categories, and at least six distinct architectural ideas (MoE, MLA, KDA, DeltaNet, Mamba/LatentMoE, sparse attention) is a lot to hold in your head at once. This post is the map, not the territory — a condensed way to navigate the full 2026 model landscape without reading fifteen separate briefings cold, plus the one terminology distinction (closed vs. open-weight vs. open-source) that trips up more procurement and architecture conversations than any benchmark score ever does.

Llama 4 gets filed under "open source" in more casual conversations than any other model in this series, and it isn't — not in the OSI sense. Meta ships Llama 4 under the Llama Community License, a genuinely permissive licence for the overwhelming majority of teams, but with one specific clause that has real teeth: if your company (or its affiliates) has more than 700 million monthly active users, you need a separate licence directly from Meta. That single number is the most consequential fact in this entire post for anyone at real scale.

Mistral Small 4's most interesting design decision isn't a parameter count — it's the word "unified." Where a lot of teams end up running a separate chat model, a separate instruct/tool-calling model, and a separate code model because each was fine-tuned and released independently, Mistral's Small 4 pairing (119B and 6B) is positioned as a single model family designed to handle that full range of tasks without switching checkpoints. Whether that consolidation is actually worth it for your stack is the real engineering question here.

Most vendors in this series ship you a model and a benchmark table. NVIDIA's Nemotron 3 ships you a model, a benchmark table, and — more unusually — a meaningfully documented account of how it was built: the data curation approach, the training recipe, and architectural choices, published in enough detail that you can reason about why it behaves the way it does rather than just that it does. For teams doing serious model customisation work, that openness about process is arguably worth more than the model itself.

OpenAI shipped GPT-5.6 to general availability on 9 July 2026 as three durable capability tiers — Sol, Terra and Luna — rather than the old mini/nano naming. That naming change is not cosmetic. It changes how you should think about model selection: you are no longer picking "the model," you are picking a tier that OpenAI can quietly upgrade underneath you on its own cadence. If you build against GPT-5.6 without understanding that distinction, you will misforecast cost and behaviour six months from now.

The newsworthy fact about gpt-oss isn't a benchmark number — it's that OpenAI, a lab whose entire commercial identity has been built around closed, API-only models, shipped genuinely Apache 2.0-licensed open-weight reasoning models. That's a strategic signal worth reading carefully, and gpt-oss-120b and gpt-oss-20b deserve evaluation on their own technical merits rather than being dismissed as a side project or overhyped as a full strategy reversal.

OpenClaw is easy to misread as "yet another chat UI for an API key." It isn't. It is a self-hosted multi-channel gateway that sits between the messaging apps you already use and an agent that can run tools, keep sessions, and talk back from your pocket — on your machine, under your rules.

Qwen 3.5 is the model to point to when someone claims "open source" and "commercially free to use without restriction" are the same category as Kimi K3 or Llama 4. They aren't — and Qwen 3.5's Apache 2.0 licence is precisely why it deserves separate treatment from its open-weight cousins. A 397-billion-parameter model with only 17B active per token, genuinely Apache-licensed, is a different legal and commercial proposition than a similarly-sized model under a custom licence, even if the benchmark numbers look comparable.

Grok 4.5 is the first model from xAI trained specifically for coding and agentic work, and the headline story — a joint-training partnership with Cursor using real developer interaction data — is genuinely unusual among frontier labs. Most vendors train on public repositories and synthetic agent trajectories; xAI is claiming an edge from live, in-editor developer behaviour. That's a real architectural difference worth understanding, and also exactly the kind of claim you should verify on your own codebase before routing production traffic to it.