Secure AI systems

Study guide for Generative AI Leader, Unit 4 · Topic 2. This is the topic's lecture in reading form — every slide's teaching, figures and worked examples, in order — followed by the official Google Cloud pages its claims rest on.

What the exam guide asks, quoted. Explaining security throughout the ML lifecycle. Identifying the purpose and benefits of Google’s Secure AI Framework (SAIF). Recognizing Google Cloud security tools and their purpose (e.g., secure-by-design infrastructure, Identity and Access Management (IAM), Security Command Center, and workload monitoring tools).

This hive's learning objectives for the topic:

Explain security risks and controls at each stage of the machine learning lifecycle.
Describe the purpose and benefits of Google's Secure AI Framework (SAIF).
Match IAM, Security Command Center, secure-by-design infrastructure and workload monitoring to what each protects.

Securing AI systems

Where AI is attacked, the framework Google offers, and the tools that act

Where? every stage of the AI lifecycle, from training data to prompts. What framework? SAIF — Google's Secure AI Framework. Which tools? IAM for access, Security Command Center for risk, secure-by-design infrastructure, and monitoring.

This topic is about protecting AI systems from malicious attacks and misuse, and it has three parts. First, where the risks sit: Google's architecture guidance asks you to integrate security into every stage of the AI lifecycle, because the attacks are different at each stage — poisoned training data is a different problem from a manipulative prompt. Second, the framework Google offers for thinking about it: SAIF, the Secure AI Framework, which Google describes as a conceptual framework for secure AI systems. Third, the tools that act: Identity and Access Management, or IAM, which controls who can do what; Security Command Center, which helps prevent, detect and respond to security issues; Google's secure-by-design infrastructure; and the monitoring that keeps the whole system visible. A leader is not expected to configure these, but is expected to know which one answers which risk.

Security throughout the ML lifecycle

Each stage has its own attack, so each needs its own control

Data: minimize what you keep, validate it before training
Training: least privilege for people and service accounts
Deployment: track posture and drift; record model provenance
Serving: screen prompts and responses for common risks
Throughout: layered defense — never one control alone

Worked example (synthetic). A retailer fine-tunes a model on scraped product reviews. Without validation, a competitor's planted reviews teach it to disparage the retailer's own brand — data poisoning, caught only at the data stage.

Google's guidance is to embed security into every stage of the AI lifecycle, from data preprocessing to your final application code, and to consider attack vectors such as data poisoning, model inversion or adversarial attacks. Take the stages in turn. At the data stage, Google asks you to adhere to data minimization for privacy, and to implement robust validation checks before you use data for training — data poisoning is an attack in which malicious data is injected into training datasets to manipulate model behaviour or degrade performance. At training, apply least privilege: permissions so that users have only the access they need, and service accounts for pipelines and deployed models with only the minimum required Identity and Access Management, or IAM, permissions; sensitive training can run on Shielded VMs or confidential computing. At deployment, track configuration drift with Security Command Center postures, and keep model provenance, which Google says provides transparency and accountability throughout the lifecycle. At serving, screen prompts and responses for common risks with Model Armor. And across all of it, use multiple defenses and never rely on a single defensive measure.

Controls laid against the lifecycle

Four stages, a control family for each, and no single wall

Figure. A band labelled AI lifecycle holds four pillars: Data (minimize, validate), Training (least privilege, isolation), Deployment (posture, provenance) and Prompts and responses (Model Armor screening). A callout reads: layered defense, never rely on one control.

Worked example (synthetic). A bank's credit assistant has strong prompt screening but its training bucket is world-readable. One strong pillar does not compensate for an empty one.

Laid out as pillars, the lifecycle view shows why one control is never enough. The data pillar holds minimization and validation before training. The training pillar holds least-privilege access for people and service accounts, and isolation such as confidential computing for sensitive workloads. The deployment pillar holds posture management — Security Command Center tracking configuration drift from your defined standards — and model provenance. The prompts-and-responses pillar holds automatic screening for common risks, which Google gives to Model Armor. The callout is Google's own instruction for prompt security, and it generalizes: use multiple defenses and never rely on a single defensive measure. An exam scenario that proposes one control as the whole answer is usually the distractor.

Risk, stage, and the control that answers it

Name the attack first; the stage and the control follow

Risk	Stage	Control Google points to
Data poisoning	Data	Validate data before training
Data exfiltration	Data, training	VPC Service Controls, org policies, fine-grained IAM
Prompt injection, jailbreaks	Serving	Model Armor screening prompts and responses
Malicious query floods	Serving	Cloud Armor against DDoS and application-layer attacks

Worked example (synthetic). A support bot is tricked by a pasted instruction into revealing internal notes. That is a serving-stage prompt attack, so prompt screening — not stronger training-data validation — is the control.

Here the same idea runs from the risk side. Data poisoning sits at the data stage, and the control is robust validation before training. Data exfiltration threatens data and training, and Google Cloud's page on the Secure AI Framework, or SAIF, says organizational policies and VPC service controls prevent exfiltration of data, along with fine-grained identity and access management controls that prevent unauthorized access. Prompt injections and jailbreaks happen at serving time, and Google says Model Armor can help customers mitigate risks such as prompt injections, jailbreaks, toxic content and sensitive data leakage. Floods of malicious queries — a denial-of-service threat — are met with Cloud Armor, which Google offers to protect against such DDoS and application layer style attacks. Matching the risk to its stage is most of the skill.

Google's Secure AI Framework (SAIF)

A framework for securing AI, so models are secure by default

A conceptual framework for secure AI systems
Addresses model risk management, security and privacy
Aims for AI that is secure-by-default when implemented
A SAIF Risk Self Assessment helps organizations adopt it
Google Cloud maps its security portfolio to SAIF's risks

Worked example (synthetic). A bank's board asks how it will know its AI is secure. The security lead proposes taking the SAIF Risk Self Assessment first, then mapping each gap to a Google Cloud control.

The second objective is the Secure AI Framework, SAIF. Google describes it as a conceptual framework for secure AI systems, taking a practical approach to addressing AI security challenges. Its purpose is stated on Google Cloud's SAIF page: it is designed to address top-of-mind concerns for security professionals — AI and machine learning model risk management, security and privacy — helping to ensure that when AI models are implemented, they are secure-by-default. Google has also created a SAIF Risk Self Assessment to support the implementation of SAIF in organizations and help them build and deploy AI systems securely. Google Cloud then looks at its own security portfolio through the lens of SAIF, pairing the risk areas the framework names with controls that answer them. For a leader, that is the benefit: SAIF turns a vague worry about AI security into named risks, a structured self-assessment, and a mapping to concrete controls.

SAIF self-assessment questions, and the controls that answer them

Each question names a risk area; each area has a control

Self-assessment question (abridged)	Risk area	Google Cloud control
Can you detect and remediate changes to training data?	Data poisoning	Data validation
Do you inventory all models, datasets and artifacts?	Unknown assets	AI Protection inventory
Do access controls prevent unauthorized reading or copying?	Model or data theft	IAM, VPC Service Controls
Are apps protected from large-scale malicious queries?	Denial of service	Cloud Armor

Worked example (synthetic). A team answers no to the inventory question: nobody can list every fine-tuned model it runs. AI Protection's inventory view becomes the first control to deploy, before any new feature.

The Risk Self Assessment for the Secure AI Framework, or SAIF, is phrased as questions, and each one points at a risk area and a control. Are you able to detect, remove and remediate malicious or accidental changes in your training, tuning or evaluation data? That is data poisoning, answered by validating data. Do you have a complete inventory of all models, datasets and related machine learning artifacts? That is the unknown-asset problem, and Security Command Center's AI Protection helps manage AI security posture across your environment, including your AI inventory. Do you have robust access controls on models, datasets and artifacts to prevent unauthorized reading or copying? That is model or data theft, answered by Identity and Access Management, or IAM, and VPC service controls. Do you protect generative applications against large-scale malicious queries? That is denial of service, answered by Cloud Armor. A no to any question is a gap with a named control.

Google Cloud's security tools and what each is for

Access, risk, infrastructure and visibility are different jobs

IAM: who can do what on which resources
Security Command Center: prevent, detect and respond to issues
Secure-by-design infrastructure: security in progressive layers
Model Armor: screens LLM prompts and responses
Cloud Monitoring: health and performance of deployed endpoints

Worked example (synthetic). An auditor asks three things: who may call the model, whether any project is misconfigured, and whether the endpoint is healthy. The answers come from IAM, Security Command Center and Cloud Monitoring in turn.

The third objective is matching Google Cloud's security tools to their purpose. Identity and Access Management, or IAM, is a tool to manage fine-grained authorization: it lets you control who can do what on which resources. When someone tries an action, IAM first checks whether they have the required permissions, and access is given by granting a role on a resource. Security Command Center is a cloud-based risk management solution that helps security professionals prevent, detect and respond to security issues; services report findings, which are records of threats or other issues found in your environments, and it can check for and correct over-permissioned accounts. Secure-by-design infrastructure is what Google builds beneath all of this: infrastructure designed to provide security through the entire information processing lifecycle, in progressive layers, down to data centers with multiple layers of physical security. For AI specifically, Model Armor proactively screens model prompts and responses. And for workload monitoring, Google points to Cloud Monitoring for visibility into the health and performance of deployed endpoints and infrastructure.

Which tool answers which question

Pick the tool from the question being asked

Tool	Question it answers	Example in an AI system
IAM	Who can do what, on which resource?	Only the pipeline's service account writes the model
Security Command Center	What is at risk or under attack?	Findings on a misconfigured AI endpoint
AI Protection	What AI assets do we have, and are they safe?	Inventory of models and datasets
Cloud Monitoring	Is the deployed workload healthy?	Endpoint latency and errors

Worked example (synthetic). A team wants to know whether any of its agents holds excessive permissions. That is a risk question across the AI inventory, so it starts in Security Command Center's AI Protection, then fixes the grant in IAM.

Reading the table by its middle column turns a list of products into a decision. If the question is who can do what on which resource, the tool is Identity and Access Management, or IAM — for example, ensuring only a pipeline's service account can write a model. If the question is what is at risk or under attack, it is Security Command Center, whose findings record threats and issues across your environments. If the question is what AI assets exist and whether they are secure, Security Command Center's AI Protection provides a view of AI security across your environment and helps you detect threats and mitigate risks to your AI inventory. And if the question is whether a deployed workload is healthy, Google points to Cloud Monitoring for the health and performance of endpoints and infrastructure. The tools work together: a risk found in Security Command Center is often fixed with an IAM change.

Screening on the way in and on the way out

Model Armor inspects the prompt and the response

Loading Diagram...

Figure 1 — Mermaid diagram

Figure: A left-to-right flowchart: a user prompt goes to Model Armor, which inspects it; the prompt or a sanitized prompt goes to the LLM, which generates a response; Model Armor inspects the response; the response or a sanitized response goes back to the user.

Worked example (synthetic). A user pastes a document hiding an instruction to ignore all rules. Model Armor flags the prompt before the model sees it, and also catches a customer's phone number before it leaves in a response.

Model Armor deserves its own picture because it acts at the point where generative AI differs most from older software. Google describes it as a service designed to enhance the security and safety of AI applications, working by proactively screening prompts and responses. The data flow is two-sided. The prompt goes to Model Armor first, which inspects it for potentially sensitive or malicious content; the prompt, or a sanitized version, then goes to the model. The model's response comes back through Model Armor, which inspects it in turn, and the response, or a sanitized version, goes to the user. In Google's words, Model Armor filters both input and output to prevent the model from exposure to, or generation of, malicious or sensitive content — which is how it addresses prompt injection on the way in and sensitive data leakage on the way out.

What this topic actually tests

Find the stage, name the risk, choose the control

Which stage? data, training, deployment or serving. Which risk? poisoning, exfiltration, prompt attacks, query floods. Which framework? SAIF's self-assessment names the gaps. Which tool? IAM for access, Security Command Center for risk, Model Armor for prompts, Cloud Monitoring for health.

Close by running the three objectives together. Find the stage of the AI lifecycle where the risk sits — data, training, deployment or serving — because Google asks you to integrate security into every stage. Name the risk: data poisoning, exfiltration, prompt injection, or floods of malicious queries. Use SAIF, the Secure AI Framework, to make that systematic: its Risk Self Assessment asks the questions, and Google Cloud maps the answers to controls. Then choose the tool by the question it answers — Identity and Access Management, or IAM, for who can do what; Security Command Center for what is at risk; Model Armor for screening prompts and responses; Cloud Monitoring for workload health — on infrastructure designed with security in progressive layers. And never rely on a single defensive measure. The next topic turns from security to responsibility.

Unit 4.2 study guide — Secure AI systems