Artificial intelligence models are improving at a remarkable pace. A model released only six months ago can quickly appear less capable, less efficient and less reliable than a newer generation. It is tempting to assume that every new model is smarter because it contains more parameters — but modern frontier systems such as Claude Fable 5, Claude Opus 5, GPT-5.6, Gemini 3.6 Flash and Kimi K3 are improving through a much broader stack of advances.

Introduction​

However, this progress is often misunderstood.

Although scale remains important, modern frontier systems improve through a combination of:

• better and more carefully balanced training data;
• more efficient model architectures;
• reinforcement learning for reasoning and tool use;
• adaptive inference-time computation;
• long-term memory and context management;
• multimodal understanding;
• self-verification and error correction;
• autonomous interaction with software;
• multi-agent coordination;
• improved safety and reliability;
• better intelligence per token, dollar and second.

As of July 29, 2026, no single model is universally the smartest. Claude Fable 5 remains exceptionally strong at very long, autonomous assignments, while Claude Opus 5 reaches near-Fable capability at lower cost. GPT-5.6 Sol is especially competitive in coding, tool use, efficiency and multi-agent execution. Gemini 3.6 Flash emphasises multimodal processing, computer use and long-context efficiency. Kimi K3 demonstrates how open-weight models and new sparse architectures are approaching the proprietary frontier. (Anthropic)

The most important change is therefore not that AI models have become better at producing text.

They are becoming better at understanding objectives, planning work, taking actions, checking results, learning from intermediate evidence and completing entire workflows.

---

1. What Does It Mean for an AI Model to Become "Smarter"?​

There is no single scientific measurement called "smartness."

A model may be excellent at mathematics but weaker at writing. Another may be brilliant at analysing documents but unreliable when operating software. A coding model may solve difficult programming tasks while struggling with business judgment.

A more useful representation of practical AI capability is:

$$\mathrm{Practical\ Intelligence} = f(K, R, P, T, M, V, S, E, A)$$

where:

• (K) = knowledge;
• (R) = reasoning ability;
• (P) = planning;
• (T) = tool-use capability;
• (M) = memory and context management;
• (V) = verification;
• (S) = safety and reliability;
• (E) = computational efficiency;
• (A) = adaptability.

An older language model might possess considerable knowledge but still perform poorly on a real assignment because it cannot plan, use tools or verify its conclusions.

A modern frontier model combines these abilities. It can receive an objective such as:

Analyse this company's financial performance, review its market position, identify commercial risks, build a forecast and prepare an executive presentation.

The model must then decide:

1. Which documents should be examined?
2. Which figures should be extracted?
3. Which calculations are necessary?
4. Which external sources should be consulted?
5. Which assumptions are uncertain?
6. Which risks deserve attention?
7. How should the final answer be presented?
8. How can the output be checked before delivery?

This is much closer to professional work than ordinary question answering.

---

Part I: The Foundations of Model Improvement

2. Scaling the Model, Data and Training Compute​

The first major source of improvement was straightforward scaling.

Researchers observed that language-model performance often follows predictable power-law relationships with:

• the number of model parameters;
• the number of training tokens;
• the amount of computing power used during training.

A simplified relationship can be expressed as:

$$L(N,D,C) \propto N^{-\alpha} + D^{-\beta} + C^{-\gamma}$$

where:

• (L) is prediction loss;
• (N) is the number of parameters;
• (D) is the amount of training data;
• (C) is training compute;
• (\alpha,\beta,\gamma) are empirically observed scaling coefficients.

As parameters, data and compute increase, training loss generally decreases. Lower prediction loss often leads to better language understanding, knowledge retrieval and downstream performance. Early scaling-law research showed that these relationships could remain predictable across many orders of magnitude. (Kaplan et al., arXiv)

However, simply building the largest possible model is not necessarily efficient.

DeepMind's Chinchilla research demonstrated that many early large models were undertrained: they had many parameters but had not seen enough high-quality training tokens. Chinchilla, with 70 billion parameters, outperformed substantially larger models because its parameter count and training-data volume were more appropriately balanced for the available compute. (Hoffmann et al., arXiv)

This changed the industry's strategy from:

$$\text{More parameters at any cost}$$

to:

$$\text{Optimal balance of parameters, data and compute}$$

Modern model developers therefore ask:

• How large should the model be?
• How many tokens should it process?
• What mixture of data should be used?
• How many times should important data be repeated?
• When does additional training stop producing worthwhile gains?
• How should the compute budget be divided between pretraining and post-training?

This is the first reason newer models can outperform older ones without necessarily being dramatically larger.

---

3. Better Data Is Becoming More Important Than More Data​

Early language models learned from huge collections of internet text. Increasing the data volume initially produced substantial improvements.

But internet data has limitations:

• duplication;
• spam;
• misinformation;
• low-quality writing;
• outdated material;
• toxic or biased content;
• generated content copied from other models;
• insufficient examples of expert reasoning;
• limited coverage of specialised professional workflows.

Modern training pipelines therefore focus increasingly on data quality, composition and curriculum design.

A frontier training dataset may combine:

• books and academic material;
• source code;
• mathematical proofs;
• scientific papers;
• business and financial documents;
• diagrams, images, audio and video;
• human demonstrations;
• synthetic reasoning problems;
• tool-use trajectories;
• computer-interaction recordings;
• examples of both successful and failed attempts.

The training mixture matters because different data types create different capabilities.

For example:

$$\text{More code data} \rightarrow \text{better programming patterns}$$ $$\text{More verified mathematics} \rightarrow \text{stronger formal reasoning}$$ $$\text{More tool-use trajectories} \rightarrow \text{better agent behaviour}$$ $$\text{More multimodal data} \rightarrow \text{better understanding of images, charts and interfaces}$$

The most valuable training examples are increasingly those where correctness can be verified.

A generated program can be compiled and tested. A mathematical answer can be checked. A database query can be executed. An interface can be rendered and inspected. This makes it possible to create stronger learning signals than subjective human preference alone.

---

4. From Pretraining to Post-Training​

Pretraining teaches a model to predict the next token:

$$P(x_t \mid x_1,x_2,\ldots,x_{t-1})$$

This objective allows the model to learn language, facts, patterns and some forms of reasoning. But a pretrained model is not automatically a helpful assistant.

It may:

• continue a user's text instead of answering;
• ignore instructions;
• produce unsafe responses;
• become excessively verbose;
• fail to admit uncertainty;
• follow misleading prompts;
• generate plausible but incorrect information.

Post-training converts the raw model into a useful system.

A simplified modern pipeline may contain:

$$\text{Pretraining} \rightarrow \text{Supervised fine-tuning} \rightarrow \text{Preference optimisation} \rightarrow \text{Reasoning RL} \rightarrow \text{Tool-use RL} \rightarrow \text{Safety training}$$

Supervised fine-tuning​

The model is shown examples of desirable behaviour:

$$(\text{instruction},\ \text{high-quality answer})$$

It learns how to follow instructions, structure answers and interact conversationally.

Preference training​

Human reviewers or specialised evaluators compare model responses. The system learns which outputs are:

• more helpful;
• more accurate;
• clearer;
• safer;
• better aligned with user intent.

Reinforcement learning​

The model performs tasks and receives rewards based on outcomes.

For a coding task:

$$r = \begin{cases} 1, & \text{tests pass} \\ 0, & \text{tests fail} \end{cases}$$

For a mathematical problem:

$$r = \begin{cases} 1, & \text{answer is correct} \\ 0, & \text{answer is incorrect} \end{cases}$$

Repeated optimisation encourages strategies that are more likely to produce successful outcomes.

---

Part II: How Models Learned to Reason

5. Reinforcement Learning for Reasoning​

One of the most important breakthroughs has been training models specifically to solve problems through multiple reasoning steps.

Instead of producing the first likely answer, a reasoning model may learn to:

1. Decompose the problem.
2. Explore a possible solution.
3. Detect a contradiction.
4. Backtrack.
5. Try an alternative.
6. Verify the result.
7. Produce the final answer.

DeepSeek's R1 research demonstrated that reinforcement learning with verifiable rewards could encourage behaviours such as reflection, alternative exploration and self-correction—even when complete human-written reasoning demonstrations were not provided. The research also showed that reasoning-focused reinforcement learning alone could produce readability problems, which were addressed through additional supervised data and multi-stage training. (DeepSeek-R1, arXiv)

The key change is:

$$\text{Older model:} \quad \text{Question}\rightarrow\text{Immediate answer}$$ $$\text{Reasoning model:} \quad \text{Question}\rightarrow\text{Explore}\rightarrow \text{Check}\rightarrow\text{Revise}\rightarrow\text{Answer}$$

This does not mean every visible reasoning explanation perfectly represents the model's internal computation. Research continues to show that generated explanations can sometimes be post-hoc or decorative rather than causally faithful. Reasoning traces should therefore not automatically be treated as reliable evidence of how the answer was reached. (arXiv)

The real objective is not to produce a long explanation. It is to increase the probability of reaching a correct and useful result.

---

6. Test-Time Compute: Giving Harder Problems More Thinking​

Traditional language models generally use approximately the same amount of neural-network computation for every generated token.

Modern reasoning systems can allocate different levels of inference effort depending on the problem.

Conceptually:

$$C_{\text{inference}} = f( \text{task difficulty}, \text{uncertainty}, \text{value}, \text{risk} )$$

A simple request such as rewriting a sentence may require little additional reasoning.

A difficult request involving software architecture, financial modelling or scientific research may justify:

• longer internal deliberation;
• multiple candidate solutions;
• additional searches;
• repeated simulations;
• more tool calls;
• parallel agents;
• verification passes.

This is called test-time scaling or inference-time scaling.

Research indicates that increasing test-time compute can improve reasoning performance, but no single strategy is optimal for every model and task. The best approach depends on model type, difficulty and compute budget. A harder problem may benefit from longer reasoning or multiple sampled attempts, while easy problems may gain little from additional computation. (arXiv)

This creates a second scaling dimension:

$$\text{Training-time scaling} + \text{Inference-time scaling}$$

The model can be trained once but use additional computation when a particularly valuable problem is presented.

---

7. Adaptive Reasoning Levels​

Current frontier models increasingly offer reasoning or effort controls.

Rather than maintaining a single operating mode, the system can expose configurations such as:

• low effort;
• medium effort;
• high effort;
• maximum effort;
• multi-agent or ultra mode.

The aim is to move along a quality-cost-latency frontier:

$$\text{Higher effort} \Rightarrow \begin{cases} \text{greater potential accuracy} \\ \text{more tokens} \\ \text{higher cost} \\ \text{longer processing} \end{cases}$$

GPT-5.6, for example, provides multiple capability levels and an ultra configuration that coordinates agents across parallel workstreams. Claude Opus 5 and Fable 5 similarly allow users or systems to vary reasoning effort according to the task. (OpenAI)

The long-term direction is likely to be automatic compute allocation.

The user will provide only the objective. The system will estimate how much reasoning, searching, parallelism and verification the assignment deserves.

---

Part III: From Language Models to Agents

8. Tool Use Changes What Intelligence Means​

A language model's internal knowledge is frozen at training time. It may also be unreliable at calculations, data retrieval or interacting with current systems.

Tool use allows it to extend its capabilities.

A model may be connected to:

• web search;
• databases;
• code interpreters;
• command-line environments;
• browsers;
• spreadsheets;
• email;
• calendars;
• cloud platforms;
• enterprise applications;
• file systems;
• specialist scientific software.

The basic agent loop becomes:

$$s_t \rightarrow \text{reason} \rightarrow a_t \rightarrow o_{t+1} \rightarrow s_{t+1}$$

where:

• (s_t) is the current state;
• (a_t) is an action;
• (o_{t+1}) is the resulting observation;
• (s_{t+1}) is the updated state.

For example:

1. The model decides that it needs revenue data.
2. It queries a database.
3. It receives the results.
4. It notices missing months.
5. It runs a second query.
6. It calculates growth rates.
7. It creates a chart.
8. It checks whether the chart supports the conclusion.
9. It revises the report.

This produces a fundamental transition:

$$\text{Language generation} \rightarrow \text{environment interaction}$$

The model is no longer evaluated only on what it says. It is evaluated on what it successfully accomplishes.

---

9. Models Are Being Trained on Complete Agent Trajectories​

Early tool-using systems often relied on carefully written prompts and manually programmed workflows.

Developers had to specify:

• when to search;
• which tool to call;
• how to parse results;
• how to retry;
• when to stop.

Modern models are increasingly trained on entire trajectories:

$$\tau = (s_0,a_0,o_1,s_1,a_1,o_2,\ldots,s_T)$$

The reward may depend on whether the final task was successfully completed.

Moonshot AI's Kimi-Researcher work demonstrated end-to-end reinforcement learning for multi-step searching and reasoning. The system learned to perform repeated searches, browse sources, cross-check evidence and use code during research tasks. (Moonshot AI)

Training on trajectories teaches abilities such as:

• selecting the right tool;
• recovering from tool errors;
• avoiding unnecessary actions;
• retaining relevant findings;
• recognising when more evidence is required;
• stopping after the objective has been satisfied.

This is one reason current agents can handle less structured instructions than earlier systems.

---

10. Programmatic Tool Calling​

A major challenge for agents is context overload.

Suppose a database query returns 100,000 records. Passing every record back into the language model would be expensive and inefficient.

Modern systems increasingly allow models to write or execute small programs that:

• filter results;
• aggregate data;
• remove irrelevant information;
• retain structured intermediate state;
• decide which observations need further analysis.

Instead of:

$$\text{Tool output} \rightarrow \text{all content returned to model}$$

the system can use:

$$\text{Tool output} \rightarrow \text{programmatic filtering} \rightarrow \text{relevant evidence only}$$

OpenAI describes programmatic tool calling in GPT-5.6 as a method for processing intermediate results while reducing model round trips and unnecessary context consumption. (OpenAI)

This matters because better agents are not merely more intelligent. They are also better at managing information economically.

---

11. Multi-Agent Systems​

A single model processes tasks largely sequentially:

$$T_{\text{single}} = t_1+t_2+t_3+t_4$$

A multi-agent system can divide independent tasks:

$$T_{\text{parallel}} \approx \max(t_1,t_2,t_3,t_4) + t_{\text{coordination}}$$

For example, a consulting research assignment may use:

• Agent 1: market analysis;
• Agent 2: financial analysis;
• Agent 3: competitor research;
• Agent 4: regulatory analysis;
• Lead agent: synthesis and quality control.

The benefit is not only speed. Multiple agents can provide:

• diverse approaches;
• independent verification;
• specialised roles;
• adversarial review;
• reduced dependence on one reasoning path.

GPT-5.6's ultra configuration coordinates multiple agents for complex work, while Claude's dynamic workflows and Qwen's agent-team features similarly reflect movement toward coordinated parallel execution. (OpenAI)

However, more agents do not automatically produce better outcomes.

Multi-agent systems introduce:

• communication overhead;
• duplicated effort;
• inconsistent assumptions;
• conflicting conclusions;
• higher token consumption;
• greater orchestration complexity.

The intelligence of a multi-agent system depends on its coordination protocol, not merely its agent count.

---

Part IV: Long Context, Memory and Multimodality

12. Longer Context Windows​

Early transformer models could process relatively short sequences. Modern models increasingly support hundreds of thousands or around one million tokens.

This means a model can potentially work with:

• large code repositories;
• collections of legal documents;
• financial filings;
• research libraries;
• lengthy conversations;
• enterprise operating procedures;
• months of project records.

But maximum context length is not the same as effective context understanding.

A model may accept one million tokens while failing to retrieve the most relevant fact.

Effective long-context intelligence requires:

$$\text{Long-context performance} = \text{storage} + \text{retrieval} + \text{attention} + \text{integration} + \text{reasoning}$$

Gemini 3.6 Flash supports text, images, audio and video with a context window of up to one million tokens. Google's reported evaluations also show a substantial improvement over Gemini 3.5 Flash on its million-token long-context test. (Google DeepMind)

Kimi K3 and Claude Fable 5 similarly target long-context and long-horizon workloads, although their architectural and product implementations differ. (Kimi K3, arXiv)

---

13. Persistent Memory​

Context exists only within a particular interaction. Memory is information retained across tasks or sessions.

A model may use memory to store:

• user preferences;
• project objectives;
• architectural decisions;
• discovered constraints;
• completed work;
• failed approaches;
• unresolved questions;
• monitoring rules.

A useful memory system distinguishes among:

Working memory​

Information needed for the current step.

Episodic memory​

Records of previous actions and events.

Semantic memory​

Stable facts and concepts learned about the project.

Procedural memory​

Instructions describing how recurring tasks should be performed.

A capable agent should not simply accumulate everything. It must decide:

$$\text{What should be remembered?}$$ $$\text{What should be compressed?}$$ $$\text{What should be forgotten?}$$

Poor memory creates context pollution. Outdated or incorrect assumptions can persist and influence future decisions.

Fable 5's reported strengths include maintaining focus over extended assignments and using persistent notes to improve later actions. GPT-5.6 also emphasises refining memories and sustaining long-running workflows. (Anthropic)

Memory is therefore becoming an active reasoning capability rather than a passive storage feature.

---

14. Native Multimodality​

Earlier systems often used separate models for text, images, speech and video.

Modern frontier systems increasingly process multiple modalities within a shared model or closely integrated architecture.

A multimodal model may reason across:

$$\{ \text{text}, \text{image}, \text{audio}, \text{video}, \text{interface state}, \text{tool output} \}$$

This enables workflows such as:

• reading a written product requirement;
• examining a design screenshot;
• generating the interface;
• opening the rendered application;
• identifying visual differences;
• correcting the code;
• verifying the result again.

Claude Fable 5 uses vision to inspect diagrams, tables, documents and its own software outputs. Gemini models are natively multimodal across text, image, audio and video. Kimi K3 adds native vision and "vision in the loop" for iterative software and design work. (Anthropic)

The important improvement is not simply recognising an image.

It is integrating visual evidence into an extended decision-making process.

---

Part V: Architecture and Efficiency

15. Mixture-of-Experts Models​

Dense transformers activate most model parameters for every token.

If a model contains (N) parameters, the approximate computation per token is associated with a large portion of those (N) parameters.

Mixture-of-Experts architectures divide parts of the network into specialised experts:

$$E_1,E_2,\ldots,E_m$$

A router selects a subset for each token:

$$g(x) = \operatorname{TopK} \left( W_r x \right)$$

The output becomes:

$$y = \sum_{i\in\operatorname{TopK}} g_i(x)E_i(x)$$

This allows a model to possess an enormous total parameter capacity while activating only a fraction during inference.

The potential benefit is:

$$\text{Large knowledge capacity} + \text{lower active computation}$$

Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model that activates only part of its expert network for each token. Its technical report describes 896 routed experts, with 16 effectively activated, alongside 104 billion activated parameters. (Kimi K3, arXiv)

However, MoE models create difficult engineering problems:

• expert load imbalance;
• routing instability;
• network communication overhead;
• inactive or undertrained experts;
• memory distribution;
• deployment complexity.

The architecture becomes useful only when the training and infrastructure solve those problems.

---

16. More Efficient Attention​

Standard self-attention has approximately quadratic complexity with sequence length:

$$O(n^2 d)$$

where:

• (n) is sequence length;
• (d) is representation dimension.

As context becomes longer, the number of token-to-token interactions grows rapidly.

This motivates approaches such as:

• sparse attention;
• sliding-window attention;
• grouped-query attention;
• multi-query attention;
• compressed key-value memory;
• linear attention;
• recurrent memory;
• delta-based attention;
• hybrid attention architectures.

Kimi K3 uses Kimi Delta Attention and Attention Residuals to improve information flow across both sequence length and network depth. Moonshot reports that this architecture, together with its sparse expert system and training improvements, produces considerably better scaling efficiency than Kimi K2. (Kimi K3, arXiv)

The broader lesson is that progress is shifting from only asking:

How large can we make the transformer?

to asking:

How efficiently can information move through the model?

---

17. Quantisation, Caching and Inference Optimisation​

A model can improve without changing its intellectual capability if it becomes cheaper and faster to operate.

Important techniques include:

Quantisation​

Representing weights with fewer bits:

$$\text{FP16} \rightarrow \text{FP8} \rightarrow \text{FP4}$$

This reduces memory requirements and may increase throughput.

Prompt caching​

Reusing previously computed representations for repeated context.

If a long system prompt or codebase has already been processed, the system does not need to recompute everything for every request.

Key-value caching​

Retaining attention states from previous tokens during generation.

Speculative decoding​

Using a smaller model to propose tokens that a larger model verifies.

Continuous batching​

Combining requests dynamically to improve accelerator utilisation.

Disaggregated inference​

Separating prompt processing from token generation across specialised infrastructure.

These methods make advanced models practical at scale.

The new competitive metric is increasingly:

$$\frac{\text{Successfully completed work}} {\text{cost}\times\text{latency}}$$

GPT-5.6 is explicitly positioned around higher intelligence per token and stronger performance per dollar. Gemini 3.6 Flash improves token efficiency over Gemini 3.5 Flash, while Opus 5 aims to provide near-Fable capability at roughly half Fable's price. (OpenAI)

---

Part VI: Self-Verification and Reliability

18. Models Are Learning to Check Their Work​

An older model often follows:

$$\text{Generate} \rightarrow \text{Stop}$$

A stronger model may follow:

$$\text{Generate} \rightarrow \text{Test} \rightarrow \text{Inspect} \rightarrow \text{Revise} \rightarrow \text{Retest}$$

For code, verification may involve:

• compilation;
• unit tests;
• integration tests;
• static analysis;
• browser rendering;
• accessibility checks;
• performance profiling.

For research:

• source triangulation;
• date validation;
• contradiction detection;
• calculation verification;
• citation checking.

For financial work:

• reconciliation;
• formula auditing;
• sensitivity analysis;
• cross-checking totals;
• identifying inconsistent assumptions.

Claude Fable 5 and Opus 5 are described as using tools and visual inspection to check software outputs before returning them. GPT-5.6 similarly emphasises inspecting, refining and delivering completed artifacts rather than stopping at initial generation. (Anthropic)

Verification is particularly important because fluent language can hide mistakes. A confident explanation is not proof of correctness.

---

19. Process Supervision and Outcome Supervision​

Models can be evaluated in two broad ways.

Outcome supervision​

Only the final result matters:

$$R = \begin{cases} 1, & \text{correct final answer} \\ 0, & \text{incorrect final answer} \end{cases}$$

Process supervision​

Intermediate steps are also evaluated:

$$R = \sum_{t=1}^{T} r_t$$

A system may receive rewards for:

• selecting valid evidence;
• using the correct tool;
• avoiding unsupported assumptions;
• identifying a contradiction;
• performing a useful test;
• correcting an error.

Outcome supervision is easier when answers are objectively verifiable. Process supervision can provide richer feedback but requires reliable evaluation of intermediate behaviour.

Future systems will likely combine both:

$$R_{\text{total}} = \lambda_1 R_{\text{outcome}} + \lambda_2 R_{\text{process}} + \lambda_3 R_{\text{safety}} + \lambda_4 R_{\text{efficiency}}$$

The best agent is not merely the one that eventually succeeds. It should succeed reliably, safely and economically.

---

Part VII: How the Leading Model Families Are Improving

20. Claude: From Conversational Intelligence to Long-Horizon Autonomy​

Anthropic's model evolution increasingly emphasises:

• sustained reasoning;
• coding autonomy;
• professional knowledge work;
• context management;
• proactive error detection;
• alignment and safety.

Claude Fable 5 represents the high-capability end of this strategy. Anthropic reports that its advantage becomes more visible as tasks become longer and more complicated. It is designed for ambitious coding projects, complex research and multi-stage professional assignments. (Anthropic)

Claude Opus 5 illustrates a second form of progress: capability compression.

Instead of only introducing an even more capable and expensive system, Anthropic brought performance closer to Fable's level at a lower price. Opus 5 also improves upon Opus 4.8 in coding, knowledge work, computer use and scientific research. (Anthropic)

The progression can be summarised as:

$$\text{Better answers} \rightarrow \text{better collaboration} \rightarrow \text{long-running execution} \rightarrow \text{greater capability at lower cost}$$

---

21. OpenAI: From Reasoning Models to Efficient Digital Operators​

OpenAI's recent progression emphasises:

• adaptive reasoning;
• computer use;
• coding agents;
• programmatic tool coordination;
• persistent workflows;
• multi-agent execution;
• intelligence per token.

GPT-5.6 includes Sol, Terra and Luna tiers, allowing different balances among capability, cost and latency. Sol targets frontier performance, Terra provides a balanced option and Luna focuses on cost efficiency. (OpenAI)

The ultra configuration demonstrates another direction: scaling an AI system horizontally by coordinating multiple agents.

The progression is therefore not merely:

$$\text{GPT-5.5} \rightarrow \text{slightly smarter GPT-5.6}$$

It is closer to:

$$\text{Model} \rightarrow \text{model with tools} \rightarrow \text{programmable agent} \rightarrow \text{coordinated agent system}$$

This changes the role of the model from response generator to workflow executor.

---

22. Google Gemini: Multimodality, Long Context and High-Volume Efficiency​

Google's Gemini family has consistently emphasised native multimodality and long context.

Gemini 3.6 Flash supports text, images, audio and video with up to a one-million-token context window. Google positions it as a workhorse model with improved coding, knowledge-work, multimodal and token-efficiency performance over Gemini 3.5 Flash. (Google DeepMind)

This reflects a different optimisation target from simply winning the highest reasoning benchmark.

A model used throughout search, productivity applications, mobile devices, cloud services and enterprise agents must perform well across:

• large request volumes;
• multiple modalities;
• varying latency requirements;
• different hardware environments;
• long-context applications;
• tool-driven workflows.

Gemini's development therefore illustrates the importance of breadth and deployment efficiency, not only peak benchmark intelligence.

---

23. Kimi: Open-Weight Models Approaching the Frontier​

Kimi K3 demonstrates how open-weight development is becoming more competitive with proprietary systems.

Its major technical characteristics include:

• 2.8 trillion total parameters;
• a sparse Mixture-of-Experts design;
• 104 billion activated parameters;
• native vision;
• a one-million-token context window;
• Kimi Delta Attention;
• Attention Residuals;
• reinforcement learning across general, coding and agentic domains;
• multiple reasoning-effort levels.

Moonshot reports that Kimi K3 substantially improves scaling efficiency over Kimi K2 while reaching frontier-level results on coding, reasoning, vision and long-horizon agent tasks. The company also acknowledges that its overall performance still trails the most powerful proprietary systems in parts of its evaluation suite. (Kimi K3, arXiv)

Kimi's importance is therefore not necessarily that it wins every benchmark.

It demonstrates that:

$$\text{Open weights} + \text{architectural innovation} + \text{efficient sparse computation}$$

can significantly narrow the gap with closed frontier systems.

This increases competitive pressure on all model providers to improve price, openness and deployment flexibility.

---

Part VIII: Why Progress Sometimes Appears Sudden

24. Capabilities Can Compound​

Model progress often feels nonlinear because multiple incremental improvements interact.

Suppose a model improves independently in:

• planning by 15%;
• tool selection by 15%;
• error recovery by 15%;
• verification by 15%.

A long task may require all four capabilities.

A simplified completion probability is:

$$P(\text{success}) = P(P)\times P(T)\times P(R)\times P(V)$$

Old system:

$$0.70^4 = 0.2401$$

Improved system:

$$0.85^4 = 0.5220$$

Each individual capability improved by only 15 percentage points, but end-to-end task success more than doubled.

This helps explain why a newer model may suddenly complete an assignment that an older model repeatedly failed to finish.

The improvement is not necessarily one dramatic breakthrough. It can be the compounding effect of many reliability improvements.

---

25. Agent Scaffolding Amplifies Model Progress​

A model is only one component of an AI application.

Practical system performance may be expressed as:

$$P_{\text{system}} = f( P_{\text{model}}, P_{\text{tools}}, P_{\text{retrieval}}, P_{\text{memory}}, P_{\text{orchestration}}, P_{\text{evaluation}} )$$

A moderately stronger model can become dramatically more useful when paired with:

• reliable tools;
• structured memory;
• better prompts;
• clean data;
• validation loops;
• permission controls;
• task-specific evaluations.

This also means that two products using the same underlying model may perform very differently.

One may provide:

• strong repository indexing;
• good error recovery;
• isolated execution environments;
• useful planning;
• automatic testing.

Another may provide only a chat interface.

The intelligence experienced by the user is therefore the intelligence of the whole system, not only the foundation model.

---

Part IX: A Practical Comparison Across Generations

26. How an Enterprise AI Assignment Has Changed​

Consider the following objective:

Build a compliant customer-service AI system for a European bank.

A typical 2023-era model​

The model might:

• describe a generic architecture;
• suggest using a vector database;
• mention GDPR;
• generate example Python code;
• hallucinate some implementation details;
• require the user to coordinate every step.

The output might be useful as an initial brainstorm but insufficient for delivery.

A stronger 2024–2025 system​

The system might:

• analyse requirements;
• generate a detailed architecture;
• create code components;
• build a RAG pipeline;
• propose guardrails;
• design evaluation metrics;
• use external search;
• assist with debugging.

However, the user would still need to integrate and validate most elements.

A frontier 2026 agentic system​

A capable modern system could potentially:

1. Inspect the bank's existing architecture.
2. Review policies and regulatory requirements.
3. Map relevant data sources.
4. Identify personally identifiable information.
5. Create a threat model.
6. Compare technology options.
7. Build a prototype.
8. Generate infrastructure-as-code.
9. Run automated tests.
10. Evaluate retrieval and response quality.
11. Test prompt-injection resilience.
12. Calculate estimated operating cost.
13. Produce governance documentation.
14. Build an executive presentation.
15. Record unresolved risks for human decision.
16. Revise the solution after stakeholder feedback.

Human experts would still need to govern the project, approve risk decisions and validate the solution. But the AI system could execute a much larger proportion of the analysis and implementation.

The transition is:

$$\text{Assistant} \rightarrow \text{Copilot} \rightarrow \text{Agent} \rightarrow \text{Digital delivery team}$$

---

Part X: Limitations of Current Progress

27. Benchmarks Can Be Misleading​

Model companies report performance using benchmarks covering areas such as:

• coding;
• mathematics;
• science;
• tool use;
• computer operation;
• long context;
• professional knowledge work.

Benchmarks are useful, but they have limitations:

• test data may leak into training;
• benchmark tasks may become saturated;
• model providers may use different prompts or agent harnesses;
• reasoning effort may differ;
• tool access may differ;
• cost may not be included;
• scores may not predict performance on a company's actual workload.

A model that leads one coding benchmark may perform worse on:

• an unfamiliar programming language;
• a private enterprise repository;
• a complex deployment environment;
• a project with ambiguous requirements.

The correct model-selection question is not:

Which model has the highest overall score?

It is:

Which model produces the most reliable business outcome for our workload, risk tolerance, latency requirement and budget?

---

28. Hallucinations Have Not Disappeared​

Even frontier models can:

• invent facts;
• misunderstand requirements;
• cite irrelevant evidence;
• produce incorrect calculations;
• confidently defend a mistaken assumption;
• fail silently during long tasks.

Longer reasoning does not guarantee correctness. More tool calls do not guarantee better research. Multiple agents can reinforce the same error.

The probability of an undetected error remains important:

$$P(\text{harm}) = P(\text{error}) \times P(\text{not detected}) \times \text{impact}$$

High-impact applications therefore require:

• independent validation;
• constrained tool permissions;
• audit logs;
• human approval;
• source grounding;
• deterministic checks;
• rollback mechanisms.

---

29. Autonomy Creates New Risks​

A chatbot can produce harmful advice.

An autonomous agent may be able to act on that advice.

It could potentially:

• modify production systems;
• send communications;
• delete data;
• execute financial transactions;
• expose confidential information;
• call insecure tools;
• follow malicious instructions embedded in documents.

As models become more capable, safety must shift from content filtering to system-level control.

Required controls may include:

$$\text{Identity} + \text{least privilege} + \text{sandboxing} + \text{approval gates} + \text{monitoring} + \text{auditability} + \text{reversibility}$$

Anthropic, OpenAI and Google describe increasingly extensive safeguards, evaluation and red-team programmes alongside their newest models. (Anthropic)

Safety improvement is therefore itself part of model progress.

A system that is extremely capable but cannot be trusted with access to real tools has limited enterprise value.

---

Part XI: Where Model Improvement Is Going Next

30. Dynamic Intelligence Rather Than One Fixed Model​

Future AI products will increasingly route requests among different models.

A routing system may decide:

$$m^* = \arg\max_m \left[ \frac{Q(m,x)} {\lambda_1 C(m,x) + \lambda_2 L(m,x) + \lambda_3 R(m,x)} \right]$$

where:

• (Q) = expected quality;
• (C) = cost;
• (L) = latency;
• (R) = risk.

A small model may answer routine questions.

A larger model may handle strategic analysis.

A specialised coding model may implement software.

A multimodal model may inspect visual output.

A safety model may review planned actions.

The user will experience one coherent system, although many models may operate behind it.

---

31. Learning From Verifiable Experience​

Future models will increasingly improve by performing tasks in environments where results can be automatically checked.

Potential training environments include:

• software repositories;
• simulated businesses;
• scientific laboratories;
• games;
• operating systems;
• databases;
• robotics simulators;
• cybersecurity ranges;
• financial models.

The model can attempt a task, observe the consequences and receive a reward.

$$\text{Attempt} \rightarrow \text{Environment feedback} \rightarrow \text{Update}$$

This is more powerful than merely imitating static examples.

It teaches the model to discover strategies through interaction.

---

32. More Persistent and Organisational Memory​

Current memory systems are relatively limited.

Future enterprise agents may maintain:

• project histories;
• decision registers;
• stakeholder preferences;
• architectural knowledge;
• compliance obligations;
• operating procedures;
• lessons from previous failures.

The system may continuously update an organisational knowledge graph:

$$G_t=(V_t,E_t)$$

where:

• (V_t) contains entities, decisions, documents and events;
• (E_t) contains relationships;
• (G_t) changes as the organisation changes.

The challenge will be maintaining:

• accuracy;
• privacy;
• provenance;
• deletion rights;
• temporal validity;
• access controls.

A model that remembers everything without governance could become more dangerous than useful.

---

33. World Models and Embodied Intelligence​

Language models primarily learn patterns from recorded data.

World models attempt to predict how environments change after actions:

$$P(s_{t+1}\mid s_t,a_t)$$

This enables an AI system to simulate consequences before acting.

For example:

• What happens if a robot moves this object?
• What happens if the software configuration is changed?
• What happens if the price is reduced?
• What happens if production capacity is increased?
• What happens if this molecular structure is modified?

Combining language reasoning with world modelling could produce systems that are much better at planning in physical, scientific and economic environments.

---

34. AI Systems That Improve Other AI Systems​

Frontier models are already being used to assist with:

• code generation for training infrastructure;
• dataset filtering;
• synthetic-data creation;
• evaluation design;
• experiment analysis;
• architecture search;
• debugging;
• model optimisation.

This creates a possible feedback loop:

$$\mathrm{AI}_{t} \rightarrow \text{helps researchers build} \rightarrow \mathrm{AI}_{t+1}$$

The process is not fully autonomous. Human researchers still define objectives, evaluate evidence, allocate resources and manage risk.

But as models become stronger scientific and engineering collaborators, the speed of AI research itself may increase.

---

Conclusion​

Frontier models are not becoming smarter through one isolated breakthrough.

Their progress comes from a stack of mutually reinforcing improvements:

$$\begin{aligned} \text{AI progress} =&\ \text{better pretraining} \\ &+\ \text{better data} \\ &+\ \text{reasoning reinforcement learning} \\ &+\ \text{test-time computation} \\ &+\ \text{tool-use training} \\ &+\ \text{long context and memory} \\ &+\ \text{multimodal understanding} \\ &+\ \text{self-verification} \\ &+\ \text{multi-agent coordination} \\ &+\ \text{efficient architecture} \\ &+\ \text{stronger safety systems} \end{aligned}$$

Claude Fable feels exceptionally capable because it performs well on long, ambiguous and autonomous assignments. Opus 5 demonstrates that near-frontier capability can be made more economical. GPT-5.6 focuses on completing more work with fewer tokens, stronger tools and parallel agents. Gemini advances multimodal and long-context intelligence at workhorse scale. Kimi K3 demonstrates how architectural efficiency and open weights can bring frontier capabilities to a broader ecosystem.

The competition is no longer only about producing the model with the most knowledge.

It is about producing the system that can:

$$\text{Understand} \rightarrow \text{Plan} \rightarrow \text{Act} \rightarrow \text{Observe} \rightarrow \text{Verify} \rightarrow \text{Correct} \rightarrow \text{Complete}$$

That is the real direction of model improvement: from generating plausible language to reliably accomplishing complex work.

---

Further reading​

• Anthropic — Introducing Claude Opus 5
• Anthropic — Claude Fable 5 and Claude Mythos 5
• OpenAI — GPT-5.6
• Google DeepMind — Gemini 3.6 Flash model card
• Kimi K3: Open Frontier Intelligence (arXiv)
• Kaplan et al. — Scaling Laws for Neural Language Models
• Hoffmann et al. — Training Compute-Optimal Large Language Models (Chinchilla)
• DeepSeek-R1
• The Art of Scaling Test-Time Compute
• Kimi-Researcher
• Reuters — Anthropic rolls out Opus 5
• AP News — related coverage
• The Verge — Chinese open-weight models