19 posts tagged with "LLM"

Fifteen model families, three licensing categories, and at least six distinct architectural ideas (MoE, MLA, KDA, DeltaNet, Mamba/LatentMoE, sparse attention) is a lot to hold in your head at once. This post is the map, not the territory — a condensed way to navigate the full 2026 model landscape without reading fifteen separate briefings cold, plus the one terminology distinction (closed vs. open-weight vs. open-source) that trips up more procurement and architecture conversations than any benchmark score ever does.

Llama 4 gets filed under "open source" in more casual conversations than any other model in this series, and it isn't — not in the OSI sense. Meta ships Llama 4 under the Llama Community License, a genuinely permissive licence for the overwhelming majority of teams, but with one specific clause that has real teeth: if your company (or its affiliates) has more than 700 million monthly active users, you need a separate licence directly from Meta. That single number is the most consequential fact in this entire post for anyone at real scale.

Mistral Small 4's most interesting design decision isn't a parameter count — it's the word "unified." Where a lot of teams end up running a separate chat model, a separate instruct/tool-calling model, and a separate code model because each was fine-tuned and released independently, Mistral's Small 4 pairing (119B and 6B) is positioned as a single model family designed to handle that full range of tasks without switching checkpoints. Whether that consolidation is actually worth it for your stack is the real engineering question here.

Most vendors in this series ship you a model and a benchmark table. NVIDIA's Nemotron 3 ships you a model, a benchmark table, and — more unusually — a meaningfully documented account of how it was built: the data curation approach, the training recipe, and architectural choices, published in enough detail that you can reason about why it behaves the way it does rather than just that it does. For teams doing serious model customisation work, that openness about process is arguably worth more than the model itself.

OpenAI shipped GPT-5.6 to general availability on 9 July 2026 as three durable capability tiers — Sol, Terra and Luna — rather than the old mini/nano naming. That naming change is not cosmetic. It changes how you should think about model selection: you are no longer picking "the model," you are picking a tier that OpenAI can quietly upgrade underneath you on its own cadence. If you build against GPT-5.6 without understanding that distinction, you will misforecast cost and behaviour six months from now.

The newsworthy fact about gpt-oss isn't a benchmark number — it's that OpenAI, a lab whose entire commercial identity has been built around closed, API-only models, shipped genuinely Apache 2.0-licensed open-weight reasoning models. That's a strategic signal worth reading carefully, and gpt-oss-120b and gpt-oss-20b deserve evaluation on their own technical merits rather than being dismissed as a side project or overhyped as a full strategy reversal.

Qwen 3.5 is the model to point to when someone claims "open source" and "commercially free to use without restriction" are the same category as Kimi K3 or Llama 4. They aren't — and Qwen 3.5's Apache 2.0 licence is precisely why it deserves separate treatment from its open-weight cousins. A 397-billion-parameter model with only 17B active per token, genuinely Apache-licensed, is a different legal and commercial proposition than a similarly-sized model under a custom licence, even if the benchmark numbers look comparable.

Grok 4.5 is the first model from xAI trained specifically for coding and agentic work, and the headline story — a joint-training partnership with Cursor using real developer interaction data — is genuinely unusual among frontier labs. Most vendors train on public repositories and synthetic agent trajectories; xAI is claiming an edge from live, in-editor developer behaviour. That's a real architectural difference worth understanding, and also exactly the kind of claim you should verify on your own codebase before routing production traffic to it.

An LLM architecture defines how a model converts language into numbers, moves information between tokens, stores short-term contextual memory, transforms representations through neural layers, predicts or generates tokens, and scales parameters and computation.

Most modern language models belong to one or more of these families:

• Recurrent architectures: RNN, LSTM, GRU and RWKV
• Encoder-only Transformers: BERT-style models
• Decoder-only Transformers: GPT, Llama and Mistral-style models
• Encoder–decoder Transformers: T5 and BART-style models
• Sparse Mixture-of-Experts models
• Local, sparse and linear-attention models
• Retention and delta-rule architectures
• State-space models such as Mamba
• Convolutional sequence models such as Hyena
• Hybrid attention–recurrent architectures
• Diffusion language models
• Multimodal language models
• Retrieval-augmented language-model systems