Posts about the AI Playbook product and practice
View All TagsMost small businesses assume growth means ads. The durable path is usually the opposite: run a business so good that customers, employees, suppliers, and peers recommend it — then make that recommendation easy.
Enterprise generative AI engineering is no longer limited to writing prompts and connecting an application to a large language model. A production AI system must combine software engineering, governed data access, model routing, retrieval, tool execution, evaluation, monitoring, security, cost control and continuous delivery.
Databricks addresses these requirements through an integrated set of capabilities covering the complete GenAI lifecycle: querying foundation models and agents, building custom and low-code agents, connecting agents to governed tools, preparing structured and unstructured data, implementing retrieval with AI Search, deploying agents and applications, governing traffic through Unity AI Gateway, tracing with MLflow, evaluating and monitoring quality, operationalising through LLMOps, and delivering governed experiences through Genie One.
Diffusion models are generative machine-learning models that learn to create data by reversing a gradual corruption process. During training, clean data—such as an image—is progressively disturbed with Gaussian noise. A neural network then learns how to predict and remove that noise. During generation, the model begins with random noise and repeatedly denoises it until a coherent image, video, audio clip, three-dimensional object, molecular structure, or other output emerges.
Although early diffusion systems operated directly on image pixels, modern systems often work in a compressed latent space and use either a convolutional U-Net or a transformer-based denoising network. Text encoders, cross-attention, classifier-free guidance, ControlNet, LoRA adapters, sophisticated numerical solvers, quantisation, distillation, and flow-matching techniques have made diffusion models more controllable and computationally practical.
Hugging Face Diffusers provides a modular implementation of this ecosystem. It separates pretrained denoising models, schedulers, text encoders, autoencoders, adapters, and pipelines so developers can combine and optimise them independently. The library supports image, video, audio, editing, restoration, super-resolution, depth estimation, and other diffusion-related workflows.
Twenty-two thousand five hundred and eighty. That is roughly how many GPT-2-scale models (≈124M parameters) fit inside Kimi K3 (≈2.8T parameters, 2026). Scale mattered—but it is not the whole story.
A parallel evolution happened inside the architecture itself:
AI models gradually changed from systems that repeatedly search every previous token into systems that maintain, update, erase, and selectively retrieve structured internal memories.
This article combines that memory-systems interpretation with the concrete architectural worklog 22580: From GPT2 to Kimi3, Explained by ali (@waterloo_intern), including the original diagrams, plus the research path from full softmax attention → linear attention → DeltaNet → Gated DeltaNet → Kimi Delta Attention → Kimi Linear / Kimi K3.
Most people do not lose arguments because their logic is weak. They lose because a request was framed to trigger an automatic “yes”—a favour already received, a public stand already taken, a crowd already moving, a familiar face, a title, or a closing window. Understanding those frames is how you sell ethically, lead without coercion, and defend yourself when someone else is selling you.
An LLM architecture defines how a model converts language into numbers, moves information between tokens, stores short-term contextual memory, transforms representations through neural layers, predicts or generates tokens, and scales parameters and computation.
Most modern language models belong to one or more of these families:
LLM inference is expensive because the model repeatedly recomputes attention state over the same tokens. Caching attacks that redundancy at every layer of the stack: inside one decode loop (KV cache), across concurrent requests on a serving GPU (PagedAttention / prefix caching), across API calls for stable prompt prefixes (prompt caching), and finally at the application layer when the answer itself can be reused (exact / semantic response caching).
This guide synthesizes the mechanics from Sebastian Raschka’s KV-cache and architecture work, Sankalp’s PagedAttention deep dive, provider prompt-caching docs, AWS caching patterns, and application-layer guides from Machine Learning Mastery, IBM, Latitude, ngrok, and related sources into one engineering playbook.
Generative AI cost management is more complicated than ordinary cloud cost management. A conventional application might be charged according to CPU hours, memory, storage, and network usage. An LLM application can accumulate costs through input tokens, output tokens, cached tokens, reasoning tokens, embeddings, retrieval, reranking, guardrails, model evaluation, agent tool calls, retries, observability, and the infrastructure supporting the application.
The main FinOps challenge is therefore not simply:
How can we use the cheapest model?
The correct question is:
What is the lowest sustainable cost at which the application can meet its quality, latency, reliability, security, and business-outcome requirements?
Caching in a conventional application usually means storing the result of a database query or API request. In an LLM application, there are many more opportunities: repeated questions, equivalent paraphrases, identical embeddings, repeated retrieval and reranking, repeated tool calls, stable system prompts, and recurring agent planning steps.
The strongest architecture therefore does not implement one “LLM cache.” It implements a hierarchy of specialised caches across the RAG and agent pipeline.
Deploying a large language model is not simply a question of choosing between an open-source model and a commercial API. It is an enterprise risk decision involving: