Security and Privacy

Guide · Enterprise AI Solution EngineeringPage 9 of 18Overview → … → Security & privacy

Executive summary

Security belongs in discovery, not as a late gate. Threat-model user, model, prompt, retrieval, tools and output; mitigate injection and leakage; design identity, network, secrets and GDPR controls so production review is evidence of design—not a surprise.

When to use this page

• Use from discovery onward for any AI solution with enterprise data or tools.
• Use before production approval and alongside Governance.
• Revisit when adding retrieval corpora, agent tools or new regions.

Executive view

Require threat models, DPIA where needed, and named residual risk—do not accept “the model will refuse” as a control.

Decision required: Is residual risk acceptable for the proposed autonomy and data scope?

Technical view

Cover prompt injection, retrieval poisoning, tool misuse and log leakage in the threat model and evaluation suite.

Map identity to retrieval and tools; never store secrets in code, prompts, logs or notebooks.

Owners

Owner · Security lead with AI Solution Engineer

Contributors · Privacy, legal, architecture, AI engineering, platform, responsible AI

Security by design

Security should be embedded from discovery onward. Do not wait until production review.

AI threat model

Assess user, model, prompt, data, retrieval, tool, API, infrastructure, supply chain and output.

Prompt injection

Prompt injection occurs when malicious content attempts to alter system behaviour.

Controls include separating instructions from content, input classification, tool allowlists, content sanitisation, permission checks, retrieval filtering, output validation, human approval and restricted tool scope.

Sensitive data leakage

Leakage may occur through prompts, retrieved documents, model outputs, logs, traces, analytics, support tools and training datasets.

Controls include PII masking, data minimisation, encryption, restricted logging, access control, retention limits, tenant isolation and regional processing.

Identity and access

Implement SSO, MFA, RBAC, ABAC, least privilege, workload identity, service-to-service authentication, just-in-time access and privileged-access monitoring.

Network security

Consider private endpoints, VPC or VNet integration, firewall rules, egress restrictions, API gateways, WAF, DDoS protection and network segmentation.

Secrets

Never store credentials in source code, prompts, logs, configuration files or notebook outputs. Use managed secret stores and workload identities.

GDPR design questions

• What is the processing purpose?
• What is the lawful basis?
• Is personal data required?
• Is special-category data involved?
• Where will the data be processed?
• How long will it be retained?
• Can the data subject request access or deletion?
• Is automated decision-making involved?
• Is a DPIA required?

Risk

Trusting the model to refuse unsafe tool use, or logging full prompts “for debugging,” creates preventable incidents.

Control

Include injection tests in evaluation; define PII paths for input, context and logs; answer GDPR questions with legal input where needed.

Case study: financial-services assistant

Threats include malicious documents in retrieval, agents pasting account numbers into free-text prompts, and over-broad SharePoint permissions leaking into context. Mitigations: security-trimmed retrieval, Presidio before model calls, redacted logs, no training on customer prompts, and DPIA before production.

Output

Threat model covering prompt, retrieval, tools and output; injection test cases; PII path design; identity mapping; and GDPR answers with legal input where required.

Common failure modes

• Security as a late gate rather than a design input
• Logging full prompts “for debugging”
• Trusting the model to refuse unsafe tool use
• Ignoring retrieval as an injection path
• Secrets in notebooks and CI logs

Solution Engineer checklist

Solution Engineer checklist

• Threat model covers prompt, retrieval, tools and output
• Injection tests included in evaluation
• PII path defined for input, context and logs
• Identity model mapped to retrieval and tools
• GDPR questions answered with legal input where needed

Practical exercise

Write three red-team prompts: one instruction override, one retrieval poison snippet, one social-engineering request for account data. Define the expected safe behaviour for each before build starts.

Key takeaways

• Security by design from discovery—not a late gate.
• Threat-model prompt, retrieval, tools and output.
• Identity, minimisation and redacted logs are core controls.
• Models are not a substitute for allowlists and permissions.

What to do next

1. Continue with Responsible AI and governance.
2. Harden APIs with the API Security Engineering roadmap and full guide.
3. Assess adversarial risk with the AI Red Teaming roadmap and full guide.
4. Build the full programme with the Enterprise AI Security roadmap and full guide.
5. Wire evidence into Evaluation and observability.