Architecting Production-Grade AI on AWS: Security, Scale, Governance and Compliance by Design
Security, scale, governance and compliance by design
An impressive AI demonstration can be built in days. A production AI system that is safe, lawful, observable, resilient, affordable and trusted by users is a different engineering problem.
The model is only one component. The complete system also includes identity, authorization, data pipelines, retrieval, orchestration, tools, human approval, policy enforcement, evaluation, audit evidence, incident response and organisational governance. If any of those layers is weak, a highly capable model can make the overall solution less reliable rather than more valuable.
This article presents a detailed AWS reference architecture for a multi-tenant enterprise AI platform that supports conversational AI, retrieval-augmented generation, deterministic workflows and bounded agentic actions. It is designed around:
- the EU AI Act;
- the General Data Protection Regulation (GDPR);
- ISO/IEC 42001:2023 for AI management systems;
- ISO/IEC 27001:2022 for information security management systems;
- ISO/IEC 23894:2023 for AI risk management;
- the NIST AI Risk Management Framework and its Generative AI Profile;
- the OWASP Top 10 for LLM and GenAI applications;
- the AWS Well-Architected Framework, its Generative AI Lens and its Agentic AI Lens.
The objective is not to claim that an AWS service creates compliance automatically. It does not. AWS describes security and compliance as a shared responsibility: AWS secures the infrastructure of the cloud, while the customer remains responsible for the design, configuration, data, identities, applications and controls it operates in the cloud. Likewise, ISO/IEC 42001 does not replace law, and certification is performed by an independent certification body rather than by ISO. AWS Shared Responsibility Model, ISO explanation of ISO/IEC 42001
Important: This is technical and governance guidance, not legal advice. Determine the laws, regulatory guidance, sector rules, contractual requirements and AWS service terms that apply to the specific organisation, countries, data, users and intended purpose. Involve legal counsel, the data protection officer, information security, risk, compliance, product owners and affected stakeholders.