Choose an appropriate method for retrieval and indexing

Retrieval is where most grounded solutions actually fail, and it fails quietly. The model produces a fluent answer from the wrong material, and nothing in the response signals that search returned the wrong documents. This lesson covers the decisions that determine whether the right content reaches the model at all: how content gets in, how it is enriched, how it is queried, and who plans the query.

Why This Matters

Three properties make retrieval design consequential.

Retrieval failures are invisible downstream. If search returns the wrong three documents, a good model summarizes them faithfully. Groundedness scores well. The answer is wrong. Only a measurement aimed at the retrieval step — not the response — reveals it.

Ingestion decisions are expensive to reverse. Enrichment runs at indexing time: skillsets "chunk text, generate vectors, and apply other transformations" as content flows through. Adding chunking or vectorization later is not a configuration change; the documents already in the index were written without it, so the corpus must be reprocessed.

The access-control model is chosen here. Whether a user sees only what they are entitled to is decided by the knowledge source, not by anything downstream. Once a restricted document is in the context window, every later control is a mitigation rather than a boundary.

The exam tests this as a set of paired decisions — push or pull, vectors or not, classic or agentic, indexed or remote — each turning on one stated property.

Prerequisites

• What an index, a query, and a ranked result set are.
• That an embedding is a dense vector and vector search finds near neighbours in that space.
• The distinction between lexical matching (tokens) and semantic matching (meaning).
• That Azure AI Search is the retrieval engine underpinning Foundry IQ knowledge bases.

Learning Objectives

By the end of this lesson you will be able to:

1. Choose push or pull ingestion from the two documented conditions.
2. Decide whether a workload needs vectors at all, and configure integrated vectorization when it does.
3. Select among full-text, vector, hybrid, and multimodal queries, and add the semantic ranker appropriately.
4. Distinguish classic search from agentic retrieval, including their differing region support.
5. Choose indexed or remote knowledge sources, and identify which supports user-permission inheritance.

Building Blocks

The two workloads. Indexing "loads content into an index and makes it searchable. Internally, inbound text is tokenized and stored in inverted indexes, while inbound vectors are stored in vector indexes." Querying targets that populated index. The diagram separates them for clarity, but "in Azure AI Search, they're the same component operating in read-write and read-only modes."

The one format rule. "Azure AI Search can only index JSON documents." Push uploads JSON directly; pull "retrieve[s] and serialize[s] data into JSON."

Push and pull. Pull uses an indexer over a supported data source — Azure Blob Storage, Azure Cosmos DB, Microsoft SharePoint, Microsoft OneLake, and others. Push uploads from your own code. The rule: "If you don't have a supported data source, or if your content and index must be synchronized in real time, the push method is your only option."

Skillsets and enrichment. AI enrichment applies "custom or built-in skills for text, images, and layout", chunking, vectorizing, and transforming content during indexing. A custom skill calls your own endpoint from inside the skillset. Index projections control where enriched output is written.

Query types. Full-text, vector, hybrid, and multimodal — the last querying "content containing both text and images in a single multimodal pipeline". Integrated vectorization generates the embeddings so the application does not.

Relevance tuning. The semantic ranker, synonym maps, scoring configuration, filters, faceted navigation, and autocomplete — all aimed at "improv[ing] intent matching and result quality".

Classic search and agentic retrieval. Two engines on one service, differing in who plans the query.

Deep Dive

Getting content in

The push/pull decision has a default and two exceptions, and both exceptions push you the same way.

Use pull when the content lives in a supported data source and some staleness is acceptable. The indexer handles retrieval, change detection, serialization to JSON, and scheduling — work you would otherwise write and operate.

Use push when either exception applies: the source is not supported, or the index must track the source in real time. A line-of-business system with no indexer and a no-staleness requirement triggers both at once.

The operational consequence of pull is a monitoring obligation. An indexer run reporting Success with 0 documents has done exactly what it was told: change detection surfaced nothing. That is a statement about what the indexer saw, not about the index being correct — and it reaches users as omitted content, which nobody reports as an error. Ingestion health therefore needs the document count and item-level errors beside the status, not the status alone.

Do you need vectors?

The guidance here is deliberately sceptical, and it is worth quoting because the default assumption runs the other way: "Do you need vectors? LLMs and agents don't require vectors. Only use them if you need similarity search or if you have content that can be homogenized into vectors."

Vectors cost an embedding stage at ingest, storage for the vector index, and embedding of every query. They buy paraphrase matching. If your queries are identifiers, part numbers, codes, and exact phrases, that purchase returns very little — and vector search is actively weak there, because an identifier is a token to match rather than a meaning to approximate.

When you do adopt them, integrated vectorization means the service generates embeddings in-pipeline rather than the application embedding content before ingest and queries before search.

Choosing the query type

Full-text for lexical precision — identifiers, codes, exact phrases. Vector for similarity and paraphrase. Hybrid to "combine full-text search with vector search to balance precision and recall", which is the answer whenever a scenario names both query styles. Multimodal when text and images must be queried together.

Then separate retrieval from ranking. Hybrid decides what is retrieved; the semantic ranker decides what comes first. Relevance complaints frequently need the second even when the right document was already in the candidate set — so a stem complaining about ordering rather than absence is pointing at ranking.

Classic or agentic

Classic search is "an index-first retrieval model for predictable, low-latency queries. Each query targets a single, predefined search index and returns ranked documents in one request–response cycle. No LLM-assisted planning, iteration, or synthesis occurs during retrieval." Your application composes and merges.

Agentic retrieval targets a knowledge base representing a domain: "one or more knowledge sources, an optional LLM for query planning and answer synthesis, and parameters that govern retrieval behavior." Each query "undergoes planning, decomposition into focused subqueries, parallel retrieval from knowledge sources, semantic reranking, and results merging", returning an answer or raw data plus an activity log and references.

Two practical differences decide real designs. First, who writes the planning code — the service, or you. Second, regional support: the comparison table marks region restrictions No for classic and Yes for agentic, and the getting-started checklist tells you to choose a supported region when using agentic retrieval.

Indexed or remote, and the access-control question

Knowledge sources come in two kinds: "indexed sources use the same indexing and query engines as classic search, while remote sources bypass indexing and are queried live." Remote costs a live query and is never stale; indexed is fast and as current as the last run.

That choice is also the access-control decision. The design checklist asks "Do you need user-based permission inheritance?" and answers: "Remote SharePoint is designed for this scenario, but you can also inherit user permissions attached to content in Azure Blob Storage or ADLS Gen2. For all other scenarios, you can use the security filter workaround."

This matters because trimming has to happen at retrieval. Once a restricted document is in the context window, the model has it, and filtering the generated answer is a mitigation applied after the boundary was crossed.

Worked Examples

Example 1 — an engineering catalogue. Users search exact part identifiers such as BRG-6204 and also ask descriptive questions about tolerances.

Two query styles in one requirement names hybrid search. Vector alone fails on the identifier — it returns conceptually similar bearings rather than the one asked for — and a synonym map cannot match a paraphrase that shares no terms with the document. If users then complain that the right document appears fourth rather than first, that is a ranking problem: add the semantic ranker.

Example 2 — two sources, two ingestion methods. Standards live in Blob Storage and may lag by an hour; live pricing lives in a system with no indexer and must never be stale.

Blob Storage is supported and an hour of lag is acceptable, so pull. The pricing system triggers both push exceptions — unsupported source, real-time synchronization — so push is the only option. One index, two ingestion paths, chosen per source rather than per system.

Example 3 — a scanned corpus with an enrichment bug. OCR then entity recognition; entities come back empty for scanned pages but work for digital PDFs.

The asymmetry localizes it. A digital PDF carries a text layer, so the entity skill finds content in the document's own text field. A scanned page has no text until OCR produces it, and OCR writes to its own output. If the entity skill still reads the original field it receives nothing — and returns nothing, with a Success status. Skills form a chain, and the chain is expressed by wiring each skill's input to the previous skill's output.

Visual Explanations

The ingestion and query pipeline:

Classic against agentic, and where knowledge sources sit:

Common Mistakes

Adopting vectors by default. They are for similarity search. On identifier-heavy corpora they cost more than they return and match worse.

Answering a ranking complaint with re-indexing. Freshness and relevance are different problems with different fixes.

Reading indexer Success as "content arrived". Success with zero documents means change detection found nothing — check the count and item errors.

Adding enrichment after the first full ingest. Skillsets run at indexing time, so the change requires reprocessing.

Trimming permissions after retrieval. Once the document is in context, the boundary has already been crossed.

Using classic search then writing the merge logic yourself when the requirement explicitly says the retrieval layer should plan — that is what agentic retrieval is.

Forgetting agentic retrieval's regional restrictions. Classic search has none; agentic does, and the design must choose a supported region.

Practice Exercises

1. Content is in Cosmos DB (supported) and may lag 30 minutes; a second source has no indexer and must never be stale. Which ingestion method for each, and which rule decides?
2. Users search exact SKUs and ask paraphrased questions. Which query type, and why does vector-only fail?
3. An indexer reports Success with 0 documents while new files are missing from answers. What is the most consistent explanation?
4. A requirement says the retrieval layer, not the application, should plan across four collections. Which engine, and what does the response carry beyond the answer?
5. Which knowledge-source kind supports per-user permission inheritance, and what is the fallback elsewhere?

Summary & Concept Map

Retrieval design is four paired decisions. Push or pull: pull for supported sources, push when the source is unsupported or the index must be real-time. Vectors or not: only for similarity search, because they cost a stage and match poorly on identifiers. Which query: full-text for tokens, vector for meaning, hybrid for both, with the semantic ranker deciding order rather than membership. Classic or agentic: who plans the query, at the cost of regional restrictions. Underneath all four sits the access-control choice, made when you pick indexed or remote sources — and enrichment, which runs at indexing time and is therefore expensive to add late.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

Quick Note — Choose an appropriate method for retrieval and indexing

A team indexes an engineering catalogue with vector search alone because vectors sound modern. Then a user searches BRG-6204 — an exact part number — and gets a list of conceptually similar bearings, none of them the one asked for. Vectors approximate meaning; an identifier is a token to match. The fix is not a better embedding model, it is hybrid search: run both and fuse the results.

Decision anchors

Read the answers once, then cover the right-hand column and reconstruct each one from the prompt. A useful answer names the requirement, the capability that satisfies it, and the nearest alternative it rejects. If you can only recognize the answer after seeing it, retrieval is not yet secure.

Ninety-second explanation

Without notes, explain:

1. When is push the only permitted ingestion method?
2. Why does vector search alone fail on part numbers?
3. What does a knowledge base do that a single index query does not?
4. Why is adding chunking to an existing pipeline more expensive than it looks?
5. Which retrieval choice is also an access-control decision?

Then check yourself against current Microsoft Learn. Record the missing decision rule, not merely the missed product name, in your error log.

When to go deeper

Go deeper when the design turns on permission inheritance or on agentic retrieval's regional support — both are documented per-source and per-region. Start from Introduction to Azure AI Search.

Source and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026). Reviewed 2026-08-20. Microsoft Learn controls changing product contracts — availability, limits, preview status, naming, and retirement dates move, and this note is deliberately compact.

Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools

Model selection is the first architectural decision in a Foundry solution and the one most often made backwards. Teams reach for the most capable model available, discover the bill, and then tune prompts to claw back cost that a different model would never have incurred. This lesson builds a repeatable selection method: eliminate cheaply, measure late, and let the requirement — not the leaderboard — name the model.

Why This Matters

A model choice sets your cost curve, your latency floor, and your failure modes for the life of the workload. Three properties make it unusually unforgiving.

Cost scales with volume, and reasoning is invisible. Reasoning models generate reasoning tokens in addition to the tokens you can see. Those tokens "never appear in the message content, but they occupy space in the context window and are billed as output tokens." A summarization endpoint returning 200 visible words can be billed for thousands of tokens. At ten million requests a day, choosing a frontier reasoning model for a six-way classification is not a small inefficiency — it is the entire budget.

Capability you do not use is capability you still pay for. A model that can plan a multi-step research task brings that machinery to every request, including the ones that need a single label. The extra depth does not make a fixed-label classification more correct; the answer space was six options either way.

Some constraints cannot be bought off. Data residency, egress prohibitions, and genuinely offline operation are absolute. No amount of model quality satisfies a rule that says the audio must not leave the device. These constraints eliminate whole deployment families in one move, which is why they belong early in the decision rather than late.

The exam tests this as a reading skill. A well-written scenario names the deciding property in one sentence — ten million messages a day, no connectivity at the site, values feed a pricing engine — and the rest is context. Learning to find that sentence is most of the objective.

Prerequisites

Before this lesson you should be comfortable with:

• The distinction between a model and a deployment — you deploy a model and address it by the deployment name you chose, which need not match the underlying model name.
• What an embedding is: a dense vector representing meaning, used for similarity search rather than for generating an answer.
• The basic shape of a Foundry project and the fact that models are reached through a project endpoint.
• That Foundry Tools are prebuilt capabilities — Document Intelligence, Speech, Language, Vision, Content Safety, AI Search — distinct from the generative models in the catalogue.

Learning Objectives

By the end of this lesson you will be able to:

1. Apply a four-step elimination order — modality, complexity and envelope, placement, evaluation — to any selection scenario.
2. Distinguish when a small language model outperforms a frontier model on the criteria that matter.
3. Explain how reasoning tokens and reasoning_effort change the cost calculus of reasoning models.
4. Decide when model-router earns its dispatch step and when it is overhead.
5. Choose between a Foundry Tool and a generative model for a well-defined task.

Building Blocks

Frontier reasoning models. The most capable tier — multi-step reasoning, planning, and tool-calling orchestration. They deliberate internally, producing reasoning tokens that are billed as output. The reasoning_effort parameter governs how much they think, with supported values including none, minimal, low, medium, high, xhigh, and max. Crucially, effort is a request parameter, not a property of the deployment: one deployment can serve a none-effort routing call and a high-effort analysis call in the same pipeline.

Small language models (SLMs). Compact models sized for narrow, high-volume, latency-sensitive work — classification, routing, extraction against a fixed label set. The Phi-4 family is the reference example. They are not "worse" models; they are correctly sized ones.

Multimodal models. Accept images alongside text in a single request and reason over both — scenes, diagrams, charts. The right choice when the questions asked of an image are open-ended and unpredictable.

Embedding models. Return dense vectors. text-embedding-3-large is the reference. They enable similarity search and RAG grounding, and they never return a label or an answer on their own.

Image and video generation models. gpt-image-2 is generally available and supports arbitrary resolutions — both edges a multiple of 16 px, long edge up to 3,840 px (4K), aspect ratio up to 3:1. The gpt-image-1 series — gpt-image-1, gpt-image-1.5, gpt-image-1-mini — is in limited access preview and restricted to the fixed sizes 1024x1024, 1024x1536, and 1536x1024. Video generation is served by Sora 2 through an asynchronous jobs API.

Foundry Tools. Prebuilt capabilities with maintained schemas and classifiers. Document Intelligence returns extracted fields as strongly typed data — InvoiceDate as a date, SubTotal as a currency — so "normalization happens automatically without any configuration." Content Safety classifies harm categories and detects injection. AI Search performs hybrid retrieval with a semantic ranker.

model-router. A deployable model that assesses each incoming request and forwards it to a suitable model in its pool, behind a single endpoint.

Foundry Local. Runs a model on the device, for scenarios where connectivity is absent or data must not leave the hardware boundary.

Deep Dive

The elimination order

Selection is a filter, and the order matters because each step is cheaper than the next.

1. Modality. Absolute, and it eliminates whole families in one move. An embedding model cannot generate an image; an image model cannot classify text. No capability argument survives a modality mismatch, so this test costs nothing and removes the most candidates.

2. Complexity and the envelope. Now narrow within the surviving family. Ask what the task actually requires: does it need multi-step deliberation, or is it a single judgement against a small answer space? Then apply the envelope — throughput, latency target, cost per request. A narrow task at high volume with a tight latency target names a small language model without further argument.

3. Placement. Where is the model permitted to run? Data residency, egress prohibitions, sovereignty, and offline operation are constraints that can eliminate every cloud-hosted candidate regardless of tier. This step comes third because it is still free to apply and can invalidate everything above it — but it is written third because it is scenario-specific rather than universal.

4. Evaluate on your own data. Only now is there a short list worth measuring. Evaluation is the expensive step — it needs datasets, evaluator runs, and interpretation — so it goes last, against candidates that have already survived the free filters. Benchmarks comparing models on public datasets help at this stage; they do not replace measurement on your own task.

Reasoning models and the cost of thinking

The economics of reasoning models are unintuitive because the expensive part is invisible. Reasoning tokens are billed as output tokens and consume the context window, and a single request "can spend anywhere from a few hundred to tens of thousands of reasoning tokens depending on how hard the problem is."

Two consequences follow. First, token analytics that count only visible output will understate the bill; read completion_tokens_details.reasoning_tokens on a Chat Completions response, or output_tokens_details.reasoning_tokens on a Responses API response, to see what was actually spent.

Second, the output cap covers reasoning too. max_completion_tokens (Chat Completions) and max_output_tokens (Responses) both cover "reasoning tokens, visible output tokens, and formatting tokens." A cap sized for the visible answer can therefore be exhausted before any visible output is produced — and the documented result is that "you pay for input and reasoning tokens but receive no answer." Applications must check status on every response rather than treating an empty body as an empty answer. The guidance suggests reserving around 25,000 tokens for reasoning plus output while learning a workload's appetite, then tuning down.

The lever that controls the spend is reasoning_effort. Because it is a request parameter, a pipeline can set it per step — none or low on mechanical routing and classification, higher only where deliberation earns its cost.

When routing helps, and when it does not

model-router is deployed like any other model and dispatches each request to a suitable model in its pool. It earns its place when request complexity genuinely varies — the classic shape being a large majority of simple lookups and a small minority needing depth. Cost then tracks complexity without the team building and maintaining a classifier.

It stops being worth the dispatch step when traffic is homogeneous. Ten million instances of the same simple classification have no complexity variance to exploit; the router adds a hop and still forwards to a small model. Choosing the small model directly is simpler and cheaper.

Routing also interacts with governance. Because the serving model becomes a per-request property, any requirement to state which model produced a given output means capturing that from the response and storing it with the output. Reconstructing it later fails, because the pool's contents and the router's behaviour change over time.

Tool or model?

The sharpest recurring decision is not which model but whether a model at all. Reach for a Foundry Tool when the task is well-defined, a schema or classifier already exists, and you want typed, consistent output with a maintained contract. Reach for a model when the task is open-ended, the output is prose, or nothing prebuilt covers it.

The trap is that a well-prompted frontier model will often produce the right values for a task a tool was built for. What it cannot supply is the guarantee: a maintained schema, automatic type normalization, and stability when the input layout changes. A prompt shifts that maintenance onto you, and the failure mode — a renamed field that parses cleanly and feeds a wrong number downstream — is silent.

Worked Examples

Example 1 — high-volume intent classification. Ten million short customer messages a day must be sorted into six fixed categories at the lowest cost per message.

Modality: text in, label out — every text model survives. Complexity and envelope: the answer space is six options and volume dominates cost, which names a small language model. Placement: unconstrained. Evaluate: measure candidate SLMs on a labelled sample of real messages. A frontier reasoning model would spend reasoning tokens on a six-way choice ten million times; model-router would add a dispatch step to homogeneous traffic and still route to a small model.

Example 2 — invoice field extraction feeding a pricing engine. Standard supplier invoices; the downstream service rejects anything not matching its schema.

The phrase standard plus typed values moves this out of model selection entirely. Document Intelligence with prebuilt-invoice returns fields whose schemas are "defined and maintained by Microsoft," with InvoiceDate as a date and SubTotal as a currency. A generative model would often produce the right values and would offer no guarantee of the right shape.

Example 3 — offline field inspection. Inspectors dictate findings at remote sites with no connectivity, and policy forbids audio leaving the device.

Placement decides this before anything else. Both constraints eliminate every cloud-hosted candidate regardless of capability or price, so the answer is an on-device deployment. Choosing a cheaper cloud model would satisfy neither constraint — the audio still leaves the device.

Visual Explanations

The selection filter, in order:

Where a Tool short-circuits the whole tree:

Common Mistakes

Choosing by capability instead of by requirement. The most capable model is the right answer only when the task needs that capability. A stem describing a narrow, high-volume task is not asking which model is best.

Treating reasoning models as having one cost per call. Effort is per request. A pipeline that leaves every step at the default effort pays frontier deliberation prices on its routing decisions.

Sizing the output cap for the visible answer. The cap covers reasoning tokens too, so a tight cap on a reasoning model produces billed requests with empty bodies.

Reaching for model-router on uniform traffic. Routing exploits variance. With none, it is a hop.

Assuming a prompt can replace a schema. A prompt asks; a tool guarantees. When the consumer is another system rather than a person, the guarantee is the requirement.

Applying placement constraints last. Discovering after benchmarking that no cloud model is permitted wastes the benchmark. Residency and egress are free to check and eliminate the most candidates.

Confusing GA with preview when a stem says "generally available". Among image models, only gpt-image-2 is GA; the gpt-image-1 series is limited access preview on fixed sizes.

Practice Exercises

1. A workload classifies 8 million short messages daily into six categories, lowest cost per message. Which family, and which two properties decided it?
2. A team migrates a prompt-tuned workload to a reasoning model and many requests return empty bodies while still being billed. Diagnose it and name the two parameters involved.
3. Traffic is 90% simple FAQ lookups and 10% multi-step tax reasoning, on one interactive endpoint. What do you deploy, and what would provisioned throughput solve instead?
4. Extraction of vendor, date, and total from standard invoices, output consumed by a pricing service. Tool or model, and what property makes it decisive?
5. A regulated service routes through model-router and an auditor must name the model behind any archived output. What must the design do, and why does a later lookup fail?

Summary & Concept Map

Model selection is elimination, not comparison. Modality is absolute and free to check; complexity and the cost envelope pick the tier; placement can invalidate everything above it; evaluation on your own data comes last because it is the expensive step. Reasoning models are powerful and their cost is invisible — reasoning tokens are billed as output, share the output cap, and are governed per request by reasoning_effort. model-router converts complexity variance into cost savings and adds nothing to uniform traffic. And before choosing any model, ask whether a Foundry Tool already owns the task with a maintained schema.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

Quick Note — Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools

A retail team routes 10 million short support messages a day into six fixed intent categories, and puts a frontier reasoning model behind it because "accuracy matters". The bill arrives: the model spent reasoning tokens deliberating over a six-way choice, ten million times, and reasoning tokens are billed as output tokens. A small language model answers the same question at a fraction of the cost, because the task was never hard — it was just frequent. The selection error was not picking a weak model; it was never asking what the task actually required.

Decision anchors

Read the answers once, then cover the right-hand column and reconstruct each one from the prompt. A useful answer names the requirement, the capability that satisfies it, and the nearest alternative it rejects. If you can only recognize the answer after seeing it, retrieval is not yet secure.

Ninety-second explanation

Without notes, explain:

1. Which property of a task makes a small language model the right choice rather than a cheap frontier model?
2. Why are reasoning tokens a cost concern even when the visible answer is short?
3. When does model-router stop being worth its dispatch step?
4. What kind of requirement overrides every capability argument?
5. Give a task where a Foundry Tool beats a well-prompted frontier model, and say why.

Then check yourself against current Microsoft Learn. Record the missing decision rule, not merely the missed product name, in your error log.

When to go deeper

Go deeper when a scenario turns on a specific limit, SKU capability, or preview status — those move, and a compact note cannot track them. Start from Foundry Models sold by Azure for the catalogue and availability, then the individual model's page for parameters.

Source and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026). Reviewed 2026-08-20. Microsoft Learn controls changing product contracts — availability, limits, preview status, naming, and retirement dates move, and this note is deliberately compact.

Choose appropriate memory, tool, and knowledge integration services for agent solutions

An agent is a model, a set of instructions, and a set of tools. This objective is about the third part and the stores behind it: how an agent reaches knowledge, how it remembers, and how it acts. The recurring error is collapsing several distinct mechanisms into one — usually "memory" — and discovering later that none of the requirements was actually met.

Why This Matters

Agent scenarios routinely bundle three or four requirements that sound like the same thing: recall what the user told us, answer from the corporate corpus, keep a record for audit, and call the ticketing API. Each is a different mechanism with different durability, scope, and governance.

Choosing wrongly is expensive in a specific way: the design appears to work. An agent given a knowledge base instead of memory will still answer questions — it simply will not remember the user's stated preference next month. An agent whose transcripts live only in telemetry will still be observable — it simply cannot satisfy a seven-year audit. These failures surface long after the design review.

There is also a governance dimension. The tool integration you choose decides who executes the code, whose identity the downstream system sees, and who can change the tool underneath you. Those are security decisions wearing the costume of an implementation detail.

Prerequisites

• What an agent is: a model, instructions, and tools.
• That the Responses API is the single entry point behind every agent type.
• The difference between prompt agents (configuration only) and hosted agents (your code, run by Foundry).
• Basic familiarity with vector search as a grounding mechanism.

Learning Objectives

By the end of this lesson you will be able to:

1. Distinguish memory, conversation state, and knowledge and select each for its own requirement.
2. Choose among File Search, the Azure AI Search tool, and a knowledge base for grounding.
3. Select the correct tool type by asking who executes the call.
4. Explain what a toolbox provides and why its versioning matters.
5. Choose an authentication approach for a tool, including when On-Behalf-Of is required.

Building Blocks

Memory (preview). Durable recall of facts about the user across sessions. It is explicitly a preview capability — "some tools, including memory and web search, are in preview" — which means no SLA and not recommended for production. A readiness review must record that.

Conversation state. The record of what was said. store=false prevents service-side persistence; bring your own resources puts it in infrastructure you control, with "Azure Cosmos DB for conversation state" named for compliance and operational needs. Hosted agents additionally get session-level state persistence from the platform.

File Search. Augments an agent "with knowledge from uploaded files or proprietary documents by using vector search." For documents supplied to the agent, not for a governed corpus.

Azure AI Search tool. Grounds agents "with data from an existing Azure AI Search index." No re-ingestion; the owning team's tuning continues to apply.

Knowledge base (Foundry IQ). One or more knowledge sources with optional planning and synthesis, reusable across agents and permission-aware.

Tool types by executor. Built-in tools are executed by the service — web search, Code Interpreter, File Search, Azure AI Search, Azure Functions, function calling, plus preview entries such as Custom Code Interpreter, Image Generation, Browser Automation, Computer Use, Microsoft Fabric, and SharePoint. OpenAPI tools connect "to external HTTP APIs by using an OpenAPI 3.0 or 3.1 specification". Function calling defines functions where "your application executes the function and returns the result". MCP connects "to tools hosted on an MCP server endpoint… best for tools shared across multiple agents or maintained by a different team". A2A (preview) connects agents to other agents.

Toolbox. "Define a curated set of tools once, manage them centrally… and expose them through a single MCP-compatible endpoint. Any MCP-compatible agent runtime or client can consume a toolbox." It is the recommended way to give agents tools, and it is versioned: "create a new version, test it, and promote it to default when you're ready."

Structured inputs. Tool configuration such as vector_store_ids, container IDs, and MCP endpoints is fixed at agent creation by default; structured inputs "allow you to override these values at runtime without creating a new agent version".

Deep Dive

Memory is not conversation state

These are the two most-confused mechanisms, and the distinction is about purpose, not storage.

Memory answers what should the agent recall and use? It is selective and durable — a user's stated dietary restriction told once in March should shape an answer in June. It is a preview capability.

Conversation state answers what was said? It is a complete record, and it exists for retention, audit, and resumption. Its governance question is where it lives: store=false for no service-side persistence, or bring-your-own Cosmos DB when the organization must own it.

The practical test: if the requirement mentions personalization, it is memory. If it mentions retention, audit, or regulators, it is conversation state. If it mentions both, you need both.

Grounding: match the shape of the corpus

Three mechanisms, distinguished by who owns the content and how many sources.

File Search is for content the agent is handed. It builds a vector store from uploaded files with no pipeline to run. The tell in a stem is a user uploading something during a conversation.

The Azure AI Search tool is for an index that already exists. The tell is language about existing investment — maintained, tuned, already indexed by the platform team. Re-uploading such a corpus through File Search duplicates it and discards the enrichment and relevance tuning that went into it.

A knowledge base is for several governed collections that must be planned and merged by the service. The tell is multiple sources plus an explicit statement that the application should not contain the planning logic.

A common wrong answer attaches several index tools to one agent for a multi-source requirement. That makes source selection a per-question tool choice by the model, so nothing merges or ranks across collections — the agent picks one source and answers from it.

Multi-tenancy without multiple agents

A recurring design problem: one agent definition, several tenants, each grounded in its own store. The naive answers are an agent version per tenant (multiplying maintenance) or a tenant name in the prompt (which cannot restrict what the retrieval tool searches).

Structured inputs solve it. Tool properties that support runtime override include file_search.vector_store_ids, code_interpreter.container and container.file_ids, and mcp.server_label, server_url, and headers. The documented use case is exactly this: "Different users need different vector stores or files based on their context."

Tool type follows the executor

The cleanest way to choose a tool type is to ask who runs the code.

• The service runs it → a built-in tool.
• An external HTTP API, already described by a specification → an OpenAPI tool. Rewriting the spec as hand-maintained wrappers throws away documentation someone else maintains.
• Your own application runs it → function calling.
• Another team owns it, or several agents share it → MCP.

Then toolbox sits above all of them as the recommended packaging: one curated set, one MCP endpoint, consumable by any MCP-compatible runtime regardless of framework. Its versioning is the operational payoff — a breaking change becomes a new version you test and then promote, rather than an edit that takes effect the moment it is saved.

Authentication decides whose permissions apply

Supported options are "key-based access, Microsoft Entra (using the agent's managed identity or the project's managed identity), OAuth identity passthrough (On-Behalf-Of), and unauthenticated access, where appropriate."

The decisive question is whose identity the downstream system sees. With a managed identity it is the agent or project — identical for every user, so a junior employee's query can return documents they are not entitled to, with nothing in the response revealing it. With OBO, the calling user's identity flows through and their own entitlements apply at the source.

Each agent can also hold a dedicated Microsoft Entra identity, "enabling secure, scoped access to resources and APIs without sharing credentials" — and for hosted agents this is "automatic, dedicated per agent."

Worked Examples

Example 1 — three requirements, three mechanisms. An agent must recall a user's stated preferences months later, answer from a 400,000-document corpus the company already indexes, and keep a durable conversation record for audit.

Memory for the preferences. The Azure AI Search tool for the existing index — no re-ingestion, tuning preserved. Bring-your-own Cosmos DB for conversation state. The tempting wrong answer uses tracing for the audit record; traces are telemetry, sampled and retained for diagnostics, and treating an observability sink as a system of record is how a retention obligation fails.

Example 2 — one agent, three tenants. Identical behaviour, per-tenant document stores, and no appetite for three agent definitions.

Structured inputs on file_search.vector_store_ids, supplied per request. An agent version per tenant triples the maintenance; three toolboxes attached simultaneously would give the agent access to all three stores at once, which is worse than the original problem.

Example 3 — two integrations, two tool types. The agent calls a partner REST API described by an OpenAPI 3.1 specification, and separately raises tickets through code the team owns and runs.

OpenAPI tool for the partner API — the specification already exists. Function calling for ticket creation, because "your application executes the function and returns the result." MCP would be right if the ticketing tool were shared across agents or owned by another team; here it is neither.

Visual Explanations

Choosing the integration:

Packaging and identity around the tools:

Common Mistakes

Using memory as a grounding corpus. It holds facts about the user, not a document collection.

Using conversation state for personalization, or memory for audit. Retention is not recall, and recall is not a record.

Re-uploading an already-indexed corpus through File Search. It duplicates content and discards tuning.

Attaching several index tools for a multi-source requirement. Cross-source ranking never happens.

Creating an agent version per tenant. Structured inputs exist precisely to avoid this.

Choosing MCP by default. It is best for tools shared across agents or owned by another team; for one external API with a spec, an OpenAPI tool is simpler.

Using a managed identity where the user's own permissions must apply. That returns the same results to everyone — use OBO.

Treating a preview tool as production-ready. Memory and web search are preview: no SLA.

Practice Exercises

1. Distinguish memory from conversation state in one sentence each, and give the requirement wording that selects each.
2. An agent must ground on four governed collections and the firm does not want planning logic in application code. What do you use, and what is wrong with attaching four index tools?
3. One agent definition, three tenants, per-tenant vector stores. What mechanism, and which property does it override?
4. An agent calls a partner API described by OpenAPI 3.1, and separately runs ticket-creation code the team owns. Name both tool types and the rule that decides.
5. SharePoint answers must respect each signed-in user's permissions. Which authentication option, and what fails with a managed identity?

Summary & Concept Map

Agent integration is a set of small, sharp distinctions. Memory recalls facts about the user and is in preview. Conversation state is the record, and its governance question is where it lives. Grounding splits three ways by corpus ownership: files handed to the agent, an index that already exists, or several governed collections planned by the service. Tool type follows the executor — service, external API with a spec, your application, or another team — with toolbox as the recommended packaging and its versioning as the operational safeguard. And authentication is not plumbing: it decides whose permissions apply at the far end.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

Quick Note — Choose appropriate memory, tool, and knowledge integration services for agent solutions

An agent is asked to do three things: recall a user's stated preferences months later, answer from a corpus the platform team already indexes, and keep a durable record of every conversation for audit. A team that reaches for one mechanism — usually memory — satisfies none of them properly. These are three different stores with three different jobs, and the exam rewards telling them apart.

Decision anchors

Read the answers once, then cover the right-hand column and reconstruct each one from the prompt. A useful answer names the requirement, the capability that satisfies it, and the nearest alternative it rejects. If you can only recognize the answer after seeing it, retrieval is not yet secure.

Ninety-second explanation

Without notes, explain:

1. Distinguish memory from conversation state in one sentence each.
2. A corpus is already indexed and tuned by another team — what do you attach, and why not File Search?
3. What preview status must a readiness review record, and what follows from it?
4. What does toolbox versioning let you do that editing a tool in place does not?
5. Which knowledge-source kind supports per-user permission inheritance?

Then check yourself against current Microsoft Learn. Record the missing decision rule, not merely the missed product name, in your error log.

When to go deeper

Go deeper when the design depends on agent identity and authentication for a tool — key-based, Microsoft Entra via managed identity, or OAuth On-Behalf-Of passthrough, which is what makes retrieval respect the calling user's own permissions.

Source and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026). Reviewed 2026-08-20. Microsoft Learn controls changing product contracts — availability, limits, preview status, naming, and retirement dates move, and this note is deliberately compact.

Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing

The previous objective asked which model. This one asks a broader question: which service owns each part of the solution. Most real builds decompose into three or four capabilities, and the architectural error is consolidation — pushing extraction, safety, and retrieval into one long prompt and accepting approximations where guarantees were available.

Why This Matters

A generative build is rarely one call. An expense-approval pipeline needs fields out of scanned receipts, a screen against injection attempts, and answers grounded in policy documents. Each of those has a service built for it, with a maintained contract behind it.

Consolidating them into a single prompted model produces three approximations. The extraction returns strings you must parse and normalize. The safety screen is an instruction competing for attention with everything else in the context. The grounding is whatever the model remembers. All three work in demos and degrade in ways that are hard to detect, because none of them fails loudly.

The reverse error is also real: reaching for an agent runtime, a knowledge base, and a toolbox when the requirement is one prompt and one answer. Every service you add is configuration, identity, cost, and a failure mode. Selection is about matching the requirement to the smallest set of services that carries the necessary guarantees.

The exam frames this as decomposition. A scenario lists three or four requirements in separate sentences, and the answer is the combination that satisfies each one with the service designed for it.

Prerequisites

• The model families and the elimination order from the previous objective.
• That Foundry Tools are prebuilt capabilities distinct from catalogue models.
• Basic familiarity with retrieval as a concept: an index, a query, and ranked results.
• The idea that an agent differs from a model call by deciding actions at runtime.

Learning Objectives

By the end of this lesson you will be able to:

1. Map each common requirement — extraction, safety, retrieval, multimodal understanding, tool-calling — to the service that owns it.
2. Distinguish Azure AI Search from Foundry IQ and explain how they relate.
3. Decide when a workload needs Foundry Agent Service rather than a direct model call.
4. Explain why the resource type you provision constrains which services are reachable.
5. Choose between Document Intelligence and Content Understanding for a given extraction task.

Building Blocks

Azure AI Search. The retrieval and grounding engine. It supports "full-text, vector, hybrid, and multimodal queries over local (indexed) and remote content", provides AI enrichment to "chunk, vectorize, and otherwise make raw content searchable", and offers relevance tuning including the semantic ranker, synonym maps, filters, and faceted navigation. It ships two engines: classic search for single-index request-response queries, and agentic retrieval for multi-query, LLM-assisted planning.

Foundry IQ. Not a separate search product but "the managed knowledge layer that transforms enterprise content into reusable, permission-aware knowledge bases for agents", underpinned by Azure AI Search. An agent references a knowledge base for what to ground on; the knowledge base decides how.

Azure Document Intelligence in Foundry Tools. OCR and intelligent document processing — prebuilt models with schemas "defined and maintained by Microsoft", custom extraction and classification models, and layout analysis that returns tables, selection marks, and paragraph roles.

Azure Content Understanding. Processes "unstructured data of any type (image, documents, audio, video) and extracting structured insights based on pre-defined or user-defined formats". Its breadth across modalities is what distinguishes it.

Azure AI Content Safety. Four harm categories with severity ratings, Prompt Shields for direct and indirect attacks, protected material detection for text and code, and custom blocklists.

Azure Language and Azure Speech. Text analysis — PII detection, NER, language detection, text analytics for health as core capabilities — and the speech surface: real-time, fast, and batch transcription, text to speech, speech translation, and Voice Live.

Foundry Agent Service. The runtime for agents. "An agent can call tools, access external data, and make decisions across multiple steps to complete a task", and the Agent Runtime "manages conversations, tool calls, and agent lifecycle." Two agent types: prompt agents (configuration only, no code or compute to maintain) and hosted agents (your code, run by Foundry).

Deep Dive

Grounding: three shapes, three answers

Grounding questions look alike and split on how many sources and who plans the search.

One index you already maintain. Attach the Azure AI Search tool, which grounds agents "with data from an existing Azure AI Search index." Nothing is copied or re-ingested, and the owning team's enrichment and relevance tuning keep applying. Any stem describing existing, tuned investment is testing whether you point at it rather than rebuild.

Files handed to the agent. Use File Search, which augments agents "with knowledge from uploaded files or proprietary documents by using vector search." This is for documents a user supplies during a conversation, not for a governed enterprise corpus.

Several governed collections, planned by the service. Use a knowledge base. Each query "undergoes planning, decomposition into focused subqueries, parallel retrieval from knowledge sources, semantic reranking, and results merging", and the response carries "an activity log and references" alongside the answer. The distinguishing property is that planning and merging happen in the service rather than in your application code.

The trap in this area is answering a multi-source requirement with several index tools attached to one agent. That pushes source selection into the model's per-question tool choice, so nothing ever merges or ranks across collections — the agent picks one and answers from it.

Extraction: Document Intelligence or Content Understanding

Both extract structure from unstructured input, and they split on modality breadth and schema ownership.

Document Intelligence is documents and images, with a deep model family: prebuilt-read for text and searchable PDF, prebuilt-layout for tables, selection marks, and paragraph roles, and per-document-type prebuilt models whose field schemas Microsoft maintains. Its output is strongly typed — dates as dates, amounts as currency — which is why it fits pipelines feeding another system.

Content Understanding is broader: any modality, into "pre-defined or user-defined formats". It also offers standard and pro modes, where pro "is designed for advanced use cases that require multi-step reasoning and complex decision-making" and can reason over input content and reference data together.

Choose Document Intelligence when the input is documents and a maintained schema exists. Choose Content Understanding when the input spans modalities, or when the output shape is one you are defining, or when the task is a judgement rather than an extraction.

Agent or model call?

The line is definitional. "Unlike a simple chatbot that only generates text, an agent can call tools, access external data, and make decisions across multiple steps to complete a task." If nothing must be decided after the request arrives — one prompt, one answer — a direct model call is simpler, cheaper, and easier to reason about.

Requirements that genuinely indicate an agent: choosing among tools at runtime, acting and then reacting to the result, maintaining conversation state across turns, or coordinating other agents. Requirements that do not: schema-valid output (that is structured outputs on a plain call), safety screening (that is content filtering on the deployment), or low cost at volume (which argues against agent overhead).

The resource type constrains everything

A subtle but decisive planning point. "A Foundry resource provides unified access to models, agents, and tools" through a project endpoint of the form https://<resource-name>.services.ai.azure.com/api/projects/<project-name>. By contrast, "an Azure OpenAI resource provides only the /openai/v1 endpoint."

That means a team who provisioned an Azure OpenAI resource and later wants file search, code interpreter, or agents cannot get there by configuration — the surface is absent, not disabled. No client version, API version, or setting reaches an endpoint the resource does not have. Service selection therefore includes a provisioning decision made early and awkward to reverse.

Worked Examples

Example 1 — expense approval. Three requirements: key-value pairs from scanned receipts; screening for jailbreak and injection attempts; answers grounded in procurement policy with hybrid retrieval and semantic ranking.

Three requirements, three services: Document Intelligence (prebuilt-receipt, prebuilt-invoice), Content Safety (Prompt Shields plus harm filters), Azure AI Search (hybrid retrieval, semantic ranker). Options offering networking components, storage and analytics services, or perception tools mismatched to the tasks are testing whether you recognise the Foundry Tools surface at all.

Example 2 — a claims assistant. The team needs similarity search over product descriptions, and separately an assistant that decides which of several tools to call across a multi-step task.

Similarity search is Azure AI Search with vector queries — remembering that vectors are adopted for a reason: "LLMs and agents don't require vectors. Only use them if you need similarity search." The multi-step tool decision is Foundry Agent Service, because something must be decided at runtime. Neither service substitutes for the other: an agent uses retrieval, it does not provide it.

Example 3 — supplier evidence packs. Certificates, photographs of installed equipment, and recorded site interviews must be reduced to something an assistant can reason over, with specific values also going to a register.

The modality span decides it: Content Understanding, which handles image, documents, audio, and video. Document Intelligence would cover the certificates and photographs and not the interviews. And because two consumers need two shapes, the analyzer should emit both a readable representation for the assistant and typed fields for the register — from one analysis, not two.

Visual Explanations

Requirement to service:

How the grounding pieces stack:

Common Mistakes

Consolidating three requirements into one prompt. A prompt asks; a service guarantees. The failure is silent and shows up downstream.

Adding an agent runtime for a single-turn task. If nothing is decided after the request arrives, the runtime is overhead.

Attaching several index tools instead of a knowledge base. Source selection becomes a per-question tool choice, so cross-source ranking never happens.

Re-uploading a corpus that is already indexed. File Search is for files handed to the agent; an existing tuned index is reached with the Azure AI Search tool.

Assuming vectors are mandatory for grounding. They are adopted when you need similarity search, and they cost an embedding stage, storage, and pipeline.

Treating Content Understanding and Document Intelligence as interchangeable. They overlap on documents and diverge sharply on audio, video, and judgement tasks.

Provisioning an Azure OpenAI resource for a build that will need agents. Only /openai/v1 is served; agents and platform tools are unreachable by construction.

Practice Exercises

1. A build needs invoice fields, injection screening, and hybrid retrieval over policy manuals. Name the three services and the property that selects each.
2. A team already runs a tuned 400,000-document Azure AI Search index. An agent must use it. What do you attach, and why not File Search?
3. Which requirement in a stem tells you an agent runtime is needed rather than a direct model call?
4. An evidence pack contains PDFs, photographs, and recorded interviews. Which extraction service, and what rules out the alternative?
5. A workload on an Azure OpenAI resource cannot attach code interpreter. Diagnose it.

Summary & Concept Map

Service selection is decomposition. Count the distinct requirements in the scenario, then name the service that owns each with a real guarantee behind it: Azure AI Search for retrieval and ranking, Foundry IQ for reusable permission-aware grounding, Document Intelligence for typed fields from documents, Content Understanding for any modality into a shape you define, Content Safety for harm and injection, and Foundry Agent Service when actions are decided at runtime. Resist both consolidation and sprawl — and remember that the resource type you provision decides which of these are reachable at all.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

Quick Note — Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing

An expense-approval build needs three things: fields out of scanned receipts, a screen for jailbreak attempts, and answers grounded in procurement policy. A team that reaches for one frontier model and a long prompt gets all three approximately. The same build assembled from Document Intelligence, Content Safety, and AI Search gets typed fields with a maintained schema, a purpose-built injection detector, and hybrid retrieval with a semantic ranker — three guarantees instead of three hopes.

Decision anchors

Read the answers once, then cover the right-hand column and reconstruct each one from the prompt. A useful answer names the requirement, the capability that satisfies it, and the nearest alternative it rejects. If you can only recognize the answer after seeing it, retrieval is not yet secure.

Ninety-second explanation

Without notes, explain:

1. Which service provides hybrid retrieval, and what does the semantic ranker add on top?
2. What distinguishes Content Understanding from Document Intelligence?
3. Which requirement in a stem tells you an agent runtime is needed rather than a model call?
4. Why can a build on an Azure OpenAI resource fail to attach file search or code interpreter?
5. What is Foundry IQ, and what does permission-aware buy you?

Then check yourself against current Microsoft Learn. Record the missing decision rule, not merely the missed product name, in your error log.

When to go deeper

Go deeper when the choice depends on a service's current preview status or regional availability — agentic retrieval carries region restrictions that classic search does not. Start from Introduction to Azure AI Search and the Foundry Tools overviews.

Source and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026). Reviewed 2026-08-20. Microsoft Learn controls changing product contracts — availability, limits, preview status, naming, and retirement dates move, and this note is deliberately compact.

Apply responsible AI instrumentation, including evaluators, safety evaluations, and explanation tooling

Filters block content at runtime. Evaluation tells you whether the system was going to produce that content in the first place — and whether it still behaves the way it did last month. This objective is about the measurement side of responsible AI: which evaluator answers which question, when in the lifecycle you run it, and what an adversarial probe adds that a scored dataset cannot.

Why This Matters

Three ideas separate people who pass this objective from people who guess.

Evaluators are not interchangeable. Each answers one specific question, and the giveaway is what it needs as input. An evaluator that requires ground truth cannot be used where no reference answer exists. One that compares response to retrieved context is not measuring correctness against the world.

Safety measurement is not the same as safety filtering. A filter blocks the output that already exists. A safety evaluation or a red-team run tells you the propensity — how often, under what provocation, and in which category — before real users find out.

Instrumentation is lifecycle-shaped. The same evaluator means different things at model selection, in pre-production, and against live traffic. Post-production adds two modes people confuse constantly: continuous evaluation on sampled production traffic, and scheduled evaluation against a fixed dataset.

Prerequisites

• What an evaluator is: a scorer that takes a query, response, and possibly context or ground truth.
• The difference between an AI-assisted evaluator (a judge model) and a computed metric.
• That groundedness means supported by supplied context, not true in general.
• Basic familiarity with tracing and Application Insights from the observability objective.

Learning Objectives

By the end of this lesson you will be able to:

1. Select an evaluator from a stated requirement, using its required inputs as the discriminator.
2. Distinguish Groundedness from Groundedness Pro, and know which needs no model deployment.
3. Apply the agent evaluators to tool-using systems.
4. Set evaluation_level correctly and respect the no-mixing rule.
5. Choose between continuous and scheduled evaluation, and say when red teaming is the right instrument.

Building Blocks

The three lifecycle stages. Evaluation runs at base model selection (benchmarks and leaderboards to shortlist), pre-production (your own datasets, adversarial simulation, red teaming before release), and post-production (operational metrics, continuous evaluation, scheduled evaluation, scheduled red teaming, alerts).

Evaluator families.

The two groundedness evaluators. Groundedness is AI-assisted, scores 1–5, and "requires a model deployment to act as a judge". Groundedness Pro (preview) is "powered by Azure AI Content Safety", returns a binary pass/fail with reasoning, and — the exam-relevant point — does not require a model deployment.

Document Retrieval versus Retrieval. Document Retrieval measures retrieval quality using ground truth labels — it needs a labelled reference set. Retrieval assesses "how well the retrieved chunks address the query" without one.

Evaluation level. evaluation_level accepts "turn" (the default — each exchange scored separately) or "conversation" (the thread as a whole). You cannot mix levels in a single run; a run is one level or the other.

AI Red Teaming Agent. Built on PyRIT, it "runs automated scans for content safety risks", "simulates adversarial probing", and produces scorecards over risk categories and attack strategies, so teams can measure risk before deployment. It can also be scheduled post-production.

Cluster analysis. Groups failing cases so you see the pattern rather than a list of individual low scores — the difference between "score fell 4%" and "this class of query fails".

Deep Dive

Picking an evaluator by its inputs

The reliable way to choose is to ask what the evaluator must be handed.

Nothing but the response — Coherence, Fluency. Useful for form, silent about truth.

Response plus the query — Relevance, Response Completeness, Intent Resolution.

Response plus retrieved context — Groundedness, Groundedness Pro, Retrieval. These answer "is this supported by what we gave it", which is the only question a RAG stack can honestly ask without a reference set.

A reference answer — the similarity family, and Document Retrieval. If the scenario says no labelled data exists, these are eliminated immediately.

Tool definitions and the call trace — the agent family.

That last family repays a closer look, because its members are more finely divided than most people expect. Tool Selection asks whether the right tool was chosen; Tool Input Accuracy asks whether the arguments passed to it were correct; Tool Output Utilization asks whether the agent actually used what came back; Tool Call Success asks whether the call succeeded. An agent that picks the right tool, passes a malformed argument, and then ignores the error is failing three distinct evaluators — and a scenario describing exactly that behaviour is naming one of them.

Groundedness, Groundedness Pro, and the deployment constraint

Both evaluate the same property — is the response supported by the supplied context — and differ in mechanism and output.

Groundedness is AI-assisted. A judge model scores 1–5, giving a graded signal that suits tracking a number over time. It requires a model deployment.

Groundedness Pro is preview, powered by Azure AI Content Safety, returns binary pass/fail with reasoning, and requires no model deployment. That last point is the discriminator: a scenario that rules out deploying a judge — cost, region, governance — is pointing at Pro. A scenario that wants a graded trend line is pointing at the 1–5 evaluator.

The common error is treating groundedness as factuality. It is not. A response perfectly grounded in a wrong document scores well. Groundedness measures fidelity to the context; correctness of the context is a retrieval and content problem.

Turn against conversation, and the rule you cannot break

evaluation_level defaults to "turn", scoring each exchange independently. "conversation" scores the thread as a whole, which is what you need for properties that only exist across turns — did the agent stay on task, did it carry constraints forward, did it resolve the intent by the end.

The rule to remember: you cannot mix evaluation levels in a single run. A requirement to score both per-turn quality and whole-conversation adherence means two runs, not one run with mixed settings. Answer options offering a single run with both are testing exactly this.

What red teaming adds that datasets cannot

A dataset measures behaviour on inputs you thought of. Red teaming measures behaviour on inputs an adversary thinks of.

The AI Red Teaming Agent is built on PyRIT and automates the probing: it generates adversarial prompts across risk categories, applies attack strategies, and returns a scorecard so results are comparable between runs and across model versions. Because it is scriptable it belongs in the pre-production gate, and because risk can reappear with a model or prompt change it also belongs on a schedule in production.

Note what it is not. It does not replace filters — it tells you what the filters and system prompt are letting through. And it does not replace safety evaluators on ordinary traffic: an application can be adversarially robust and still produce harmful output on innocuous prompts.

Explanation tooling: from a score to a cause

A quality number tells you something changed; it does not tell you what. Three instruments turn scores into causes.

Traces show the actual path — which tool ran, with what input, what came back, how long each span took. When an evaluator flags a response, the trace is where you find out whether the model reasoned badly or the retrieval returned nothing.

Cluster analysis groups similar failures so a pattern emerges. Fifty scattered low scores are noise; fifty low scores that all involve a date range are a defect with an address.

Evaluator reasoning — Groundedness Pro's pass/fail carries reasoning, and the AI-assisted evaluators explain their score. That text is often the fastest route to the cause.

Used together, the loop is: an alert or a scheduled run flags a drop, cluster analysis names the failing class, traces expose the mechanism, and a targeted dataset confirms the fix.

Worked Examples

Example 1 — no reference answers. A support assistant must be measured for whether answers are supported by the retrieved knowledge base. No labelled answer set exists, and governance forbids deploying an extra judge model.

Groundedness Pro — it evaluates support against context, returns binary pass/fail with reasoning, and requires no model deployment. Plain Groundedness needs a judge; the similarity family and Document Retrieval need ground truth that does not exist.

Example 2 — the agent that ignores its own tool. An agent calls the correct pricing tool, passes a malformed date range, and then answers as if the error had not happened.

Three agent evaluators name three distinct faults: Tool Selection passes, Tool Input Accuracy fails on the malformed argument, Tool Output Utilization fails because the returned error was ignored. Tool Call Success captures whether the call itself succeeded. Naming the specific evaluator matters more than concluding "the agent is wrong".

Example 3 — drift after a model upgrade. A team must detect behaviour change after a version change, and separately watch live quality.

Scheduled evaluation against a fixed dataset isolates the model as the variable — the inputs are held constant, so a score change is attributable. Continuous evaluation on sampled production traffic covers live quality but cannot separate model change from traffic change. Both, for different questions.

Visual Explanations

Evaluator choice as a function of available inputs:

The post-production loop:

Common Mistakes

Treating groundedness as factual correctness. It measures support by the supplied context.

Choosing Groundedness when a judge deployment is excluded. Pro requires none.

Choosing Document Retrieval without ground truth. It needs labels; Retrieval does not.

Collapsing the agent evaluators into one. Selection, input accuracy, output utilization, and call success are distinct.

Mixing turn and conversation levels in one run. Not permitted — run twice.

Using continuous evaluation to detect drift. Sampled traffic changes underneath you; drift needs a fixed dataset.

Treating red teaming as a substitute for filters or safety evaluators. It measures what they let through.

Scoping safety evaluation to sensitive-sounding applications only.

Practice Exercises

1. Distinguish Groundedness from Groundedness Pro on three axes, and say which is chosen when deploying a judge is ruled out.
2. Which evaluators require ground truth?
3. An agent picks the right tool, sends a bad argument, and ignores the error. Name the evaluator for each fault.
4. State the evaluation_level values and the rule about combining them.
5. Why does drift detection require a fixed dataset rather than sampled traffic?

Summary & Concept Map

Responsible AI instrumentation is evaluator selection plus lifecycle placement. Choose evaluators by required inputs: no ground truth eliminates the similarity family and Document Retrieval; support-by-context is groundedness, where Pro needs no judge deployment and returns binary pass/fail while the AI-assisted evaluator scores 1–5; tool-using systems get the agent family, whose members separate selection, input accuracy, output utilization, and call success. Set evaluation_level to turn or conversation and never mix them in one run. In production, continuous evaluation watches sampled live traffic while scheduled evaluation on a fixed dataset is what detects drift, scheduled red teaming on PyRIT probes what filters let through, and cluster analysis plus traces turn a score into a cause.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.

Configure safety filters, guardrails, risk detection, and content moderation

Content filtering is one of the few areas of Foundry with meaningful defaults, and most exam questions here turn on knowing exactly what those defaults are. Some protections are on out of the box; one important one is off; and the direction in which you are allowed to weaken them is asymmetric.

Why This Matters

Three properties make the defaults worth memorising rather than reasoning about.

A safe default is not a complete default. Harm-category filtering applies without configuration, but the shield covering instructions hidden in retrieved documents is off unless you enable it. A team running on defaults is protected against the attack that arrives in the user's prompt and exposed to the one that arrives in a supplier's PDF.

Weakening is gated in one direction only. Making filters stricter is self-service; turning them off on completions requires approval. That asymmetry tells you which direction the platform considers risky.

Two severity scales coexist. The content filtering system classifies into four levels — safe, low, medium, high — and you set a threshold there. The Content Safety API returns numeric severities that differ by modality. Questions frequently hinge on which surface is being described.

Prerequisites

• That a content filter is a configuration object applied to a model deployment.
• The difference between a prompt (input) and a completion (output).
• What prompt injection is, and that content can arrive from retrieved documents as well as from a user.
• That filters are created at the resource level and associated with deployments.

Learning Objectives

By the end of this lesson you will be able to:

1. State the default filtering configuration and what it covers.
2. Distinguish direct and indirect Prompt Shields, including their differing defaults.
3. Configure thresholds, annotate-only, and blocklists, and know which changes need approval.
4. Distinguish the content filtering severity levels from the Content Safety API scales.
5. Apply a filter to a deployment, and override it per request.

Building Blocks

The four harm categories. Content Safety "recognizes four distinct categories of objectionable content": Hate and Fairness (Hate), Sexual, Violence, Self-Harm (SelfHarm). "Classification can be multi-labeled" — one sample can be both Sexual and Violence. The harm-categories table also lists Task Adherence, which "helps ensure AI Agents consistently behave in alignment with user instructions and task objectives", identifying "misaligned tool invocations, improper tool input or output relative to user intent".

The default configuration. "The content filtering system… uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high)… The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions."

Configurability. All customers can set thresholds to Low, medium, high (strictest), Medium, high, or High only, separately for prompts and completions. No filters and Annotate only are available for prompts, and for completions "If approved" — "Approval is required for turning the content filters partially or fully off on completions."

The other filters, with their defaults.

Blocklists. "You can apply a blocklist as either an input or output filter, or both… Select one or more blocklists from the dropdown, or use the built-in profanity blocklist. You can combine multiple blocklists into the same filter."

Scope and application. "Content filters can be configured at the resource level. Once a new configuration is created, it can be associated with one or more deployments." A per-request override exists: the x-policy-id header, where "the request-level content filtering configuration will override the deployment-level configuration, for the specific API call" — though it "is not available for image input (chat with images) scenarios."

The API severity scales. Text — "supports the full 0-7 severity scale… If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6." Image — "supports the trimmed version… The classifier only returns severities 0, 2, 4, and 6." Image with text (multimodal) — "supports the full 0-7 severity scale."

Deep Dive

The exposure hiding in the defaults

The single most examinable fact in this objective is that indirect-attack Prompt Shields is off by default.

The two shields address different threats. The direct-attack shield "filters / annotates user prompts that might present a Jailbreak Risk" — the user themselves trying to break the system. The indirect-attack shield covers "Indirect Attacks, also referred to as Indirect Prompt Attacks or Cross-Domain Prompt Injection Attacks, a potential vulnerability where third parties place malicious instructions inside of documents that the generative AI system can access and process."

Any RAG or agent system ingests third-party content — supplier PDFs, web pages, email, tool responses. On a default configuration, that entire surface is unprotected while something called "Prompt Shields" is switched on, which is exactly why the misconception survives.

Enabling it has a prerequisite worth knowing: the indirect shield "Requires: Document embedding and formatting", so the documents must be presented in the documented structure for the service to distinguish content from instructions.

Thresholds, and the asymmetry of weakening

Threshold configuration is straightforward: pick which severities are filtered, separately for prompts and completions. "Content detected at the 'safe' severity level is labeled in annotation output but isn't subject to filtering and isn't configurable."

What matters is the governance asymmetry. Tightening — moving from medium, high to low, medium, high — is self-service. Loosening on completions to No filters or Annotate only "requires approval", available through the Limited Access Review for modified content filters.

Annotate only is worth understanding as a distinct mode rather than a synonym for off: it "runs the respective model and returns annotations via API response, but it will not filter content." That makes it the right choice when you need to measure prevalence before enforcing, or when an application wants to make its own decision from the annotation.

Two severity vocabularies

Confusing these produces confidently wrong answers.

The content filtering system — the thing you configure on a deployment — classifies into four levels: safe, low, medium, high. You set the threshold in those terms.

The Content Safety API — called directly — returns numeric severities, and the range depends on modality. Text supports the full 0–7 scale, optionally returning a trimmed 0/2/4/6 where "each two adjacent levels are mapped to a single level". Image supports only the trimmed scale, returning nothing but 0, 2, 4 and 6. Multimodal — image with text — supports the full 0–7.

The image restriction is the detail most often tested, and the reasoning trap is assuming the ceiling was lowered. It was not: multimodal returns 7, so nothing is being reserved. The image model simply reports at coarser resolution.

Blocklists: the enumerable case

A blocklist matches terms you can name in advance, and can be applied as an input filter, an output filter, or both, with multiple blocklists combined into one filter and a built-in profanity list available.

That makes it exactly right for a defined, enumerable set — a partner's prohibited symbol list, a set of internal identifiers that must never appear in output — and exactly wrong for an open-ended behaviour where an adversary chooses the wording. The general rule: if you can write the list down, a blocklist fits; if the attacker picks the phrasing, you need a detector.

Where filters live, and the request override

Filters are created at the resource level and associated with one or more deployments. The association is the step that changes behaviour, and skipping it is the classic mistake — a carefully built configuration that is not attached affects nothing, and testing in the playground then measures the default policy.

For cases needing per-call variation, the x-policy-id request header names a custom configuration and overrides the deployment-level one for that call. Two caveats: a configuration that does not exist returns InvalidContentFilterPolicy, and the override "is not available for image input (chat with images) scenarios", where the default filter is used.

Worked Examples

Example 1 — a poisoned supplier document. A RAG assistant summarizes supplier PDFs. One contains hidden text instructing the model to reveal its system prompt. The deployment uses the default filter configuration.

Prompt Shields for indirect attacks is off by default and must be enabled; the direct-attack shield is on but inspects the user's prompt, which never contained the payload. Harm categories do not apply — an instruction to disclose a prompt is not hate, sexual, violence, or self-harm content. A blocklist cannot anticipate the attacker's wording.

Example 2 — a surprised safety team. A platform screens uploaded images and finds the API returns only 0, 2, 4 and 6, while its text pipeline returns a finer scale.

The image model supports only the trimmed scale; text supports the full 0–7 and can optionally return trimmed. Multimodal supports the full range, so nothing is reserved for it — the image classifier simply reports at coarser resolution.

Example 3 — brand symbols and provenance. Partners require that a defined set of prohibited symbols never appears, and that generated images can later be shown to be machine-generated.

A custom blocklist for the enumerable symbol list, applied to the output filter. Content Credentials for provenance. Harm categories cannot be extended with a partner list, and protected material detects known copyrighted content, which is a different question from provenance.

Visual Explanations

What the defaults cover, and what they leave open:

Two severity vocabularies:

Common Mistakes

Assuming Prompt Shields covers retrieved content by default. The indirect shield is off.

Expecting harm categories to catch injection. They classify harmful content, not instructions.

Treating annotate-only as off. It runs the model and returns annotations without filtering.

Believing loosening is symmetric with tightening. Turning filters off on completions requires approval.

Confusing the four filtering levels with the API's numeric scale. And within the API, assuming image supports 0–7.

Creating a filter and not associating it with a deployment. The association changes behaviour.

Using a blocklist against an adversary who chooses the wording. Blocklists fit enumerable sets.

Reaching for Prompt Shields to stop a leak. Shields are inbound; leakage is an output concern.

Practice Exercises

1. State the default filtering configuration precisely — categories, threshold, and which directions.
2. A supplier PDF carries hidden instructions. Which control, what is its default, and why does the other shield not help?
3. Which filter changes require approval, and in which direction?
4. The image API returns only 0, 2, 4, 6. Explain, and say what multimodal returns.
5. A partner supplies a list of prohibited symbols and separately wants provenance on generated images. Name both controls.

Summary & Concept Map

Content moderation in Foundry is a defaults question first. Four harm categories are filtered at medium on both prompts and completions without configuration; direct-attack Prompt Shields and protected material for text and code are on; indirect-attack Prompt Shields, groundedness, and PII are off — and the indirect shield is the gap that matters for any system ingesting third-party content. Tightening is self-service and loosening completions needs approval. Filters are resource-level objects that only take effect once associated with a deployment, with x-policy-id for per-request overrides. And keep the two severity vocabularies apart: four named levels where you set thresholds, numeric scales at the API where image returns only the trimmed 0/2/4/6.

Sources and freshness

Written against current Microsoft Learn documentation for the AI-103 skills measured (16 April 2026), reviewed 2026-08-20. Microsoft Learn controls every changing product contract — availability, preview status, quotas, limits, regional support, naming, and retirement dates all move independently of this lesson. Where a scenario turns on a specific number or a GA/preview boundary, confirm it against the product's own page before relying on it.