Design and implement agent and runner infrastructure

Every job needs a machine. Choosing which machine is a cost, security and maintainability decision, and the exam tests the trade-off rather than the click-path.

The options

Microsoft-hosted agents exist only in Azure DevOps Services — they are not available in Azure DevOps Server. Azure DevOps Server therefore uses self-hosted agents. A hosted Azure DevOps Services pipeline can still deploy to an on-premises target when its agent has the required connectivity.

Why teams move to self-hosted

Three reasons recur, and they map directly to exam scenarios:

1. Software that must be preinstalled or machine-bound, for example because of its licence. Hosted jobs can install other tools during a run.
2. Direct private connectivity to a non-public endpoint or on-premises system.
3. Machine-local persistence. Caches and configuration can survive between runs on a self-hosted agent.

A Microsoft-hosted agent gets a fresh VM for every job, whose filesystem is discarded afterward. That does not prohibit caching: Cache@2 can restore a server-backed dependency cache across pipeline runs. Choose self-hosted when the state must persist locally on the machine, not merely because a dependency restore is slow.

Billing models differ in kind, not just amount

Under Azure DevOps Services' current paid parallel-jobs model, Microsoft-hosted capacity is bought as concurrency. The free hosted job is limited to 1,800 minutes per month and 60 minutes per job; paid capacity removes the monthly limit and allows up to 360 minutes per job. Earlier customers can still be on a legacy per-minute plan.

GitHub-hosted agents for Azure Pipelines are a separate per-minute PAYG preview with no free tier, and availability is still rolling out by region.

Maintainability

Self-hosted is not free after setup. You own OS and tool maintenance, workspace hygiene, private connectivity, and least-privilege machine security. Keep the agent and operating system compatible; required agent-software updates can be automatic on supported systems, while an unsupported OS still requires a machine upgrade. Microsoft recommends considering Managed DevOps Pools instead of designing a new autoscalable VM Scale Set agent pool.

Primary sources

• https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/az-400
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted
• https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/cache-v2
• https://learn.microsoft.com/en-us/azure/devops/pipelines/licensing/concurrent-jobs
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/github-hosted
• https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/migrate-from-scale-set-agents
• https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/pricing
• https://learn.microsoft.com/en-us/azure/devops/pipelines/security/misc
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v5-agent

Agents and runners — quick notes

Traps

1. "Use Microsoft-hosted with Azure DevOps Server" — impossible; hosted is Services-only.
2. "A fresh hosted job cannot reuse dependencies" — false; Cache@2 restores a pipeline cache.
3. "Microsoft-hosted is never minute-limited" — false; distinguish paid current-plan concurrency from free and legacy minute limits.

Primary sources

• https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/az-400
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted
• https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/cache-v2
• https://learn.microsoft.com/en-us/azure/devops/pipelines/licensing/concurrent-jobs
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/github-hosted
• https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/migrate-from-scale-set-agents
• https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/pricing
• https://learn.microsoft.com/en-us/azure/devops/pipelines/security/misc
• https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v5-agent

Configure alerts for events in GitHub Actions and Azure Pipelines

Routes

Either can also call out from within the run — a step that posts on failure — which is the flexible option when the condition is more specific than "the run failed".

What deserves an alert

The same discipline as everywhere else in this unit: alert on things a human must act on.

The condition that gets forgotten

A step that reports failure must itself run on failure. Every step carries an implicit succeeded(), so a notification step placed after a failing step is skipped — the same defect as publishing test results without condition: succeededOrFailed().

A failure notification that only fires on success is worse than none, because the silence is read as good news.

Route to an owner

An alert to a shared inbox nobody owns is not an alert. Route to a team with the ability to act, and make ownership explicit.

Primary sources

• https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview
• https://learn.microsoft.com/en-us/azure/azure-monitor/overview

Pipeline alerting — quick notes

The classic bug: a notification step without condition: failed() (or succeededOrFailed()) is skipped when the thing it reports on fails. Silence then reads as good news.

Analyze metrics by using collected telemetry

Usage tells you what to work on

Usage data is what makes a hypothesis testable — it is the loop that closes feature flags and A/B testing back into a decision. Without it, "we shipped it" is the end of the story rather than the middle.

Performance analysis

Work from the user inward:

1. Which operations are slow? Request duration by name, at p95 and p99.
2. What are they waiting on? Dependency telemetry — database, HTTP, queue.
3. Where inside the code? Traces and profiling.

Skipping to step 3 is the common mistake: it produces a detailed answer about something that was never the problem.

Aggregates hide people

An average conceals the tail, and a healthy overall error rate conceals a single customer failing every request. Segment — by operation, region, client version, tenant — because one broken tenant inside a 0.1% global error rate is invisible in the aggregate and total for that customer.

Correlate with deployments

Release annotations put deployments on the chart. Most performance regressions have a deployment immediately before them, and seeing the two together is the fastest available diagnosis.

Primary sources

• https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview

Analyzing telemetry — quick notes

Performance, in order: which operations are slow (p95/p99) → what are they waiting on (dependencies) → where in the code (traces).

• Skipping to code profiling answers the wrong question in detail.
• Aggregates hide people — segment by operation, region, version, tenant.
• A healthy global error rate can hide one customer failing every request.
• Correlate with release annotations — regressions usually follow a deployment.

Recommend appropriate access levels

Access levels and permissions are separate controls evaluated together. Access levels determine which web-portal features a user can access. Permissions determine which actions they may perform on the objects they can reach. Increasing an access level does not itself grant repository permission.

Azure DevOps access levels

Stakeholder fits a product owner who tracks a backlog or approves a release but does not need private-project Repos access. Existing public projects are a temporary exception: Stakeholders currently have full Repos access there. New public projects cannot be created, and existing public projects are scheduled to convert to private projects in 2027. Basic enables Repos features; repository permissions still determine actual code access.

GitHub

Use an outside collaborator when a contractor or partner needs selected organization repositories without becoming an organization member. Outside collaborators cannot join teams. With Enterprise Managed Users, this role is called repository collaborator.

Reviewing regularly

Use Microsoft Entra group rules to assign access levels where practical. Azure DevOps adjusts a user's access level when they leave the group. Regularly review the Group rules tab so assignments continue to match current needs.

Primary sources

• https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/az-400
• https://learn.microsoft.com/en-us/azure/devops/organizations/security/stakeholder-access?view=azure-devops
• https://learn.microsoft.com/en-us/azure/devops/organizations/security/access-levels?view=azure-devops
• https://learn.microsoft.com/en-us/azure/devops/organizations/billing/buy-basic-access-add-users?view=azure-devops
• https://learn.microsoft.com/en-us/azure/devops/organizations/security/about-permissions?view=azure-devops
• https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/assign-access-levels-by-group-membership?view=azure-devops
• https://learn.microsoft.com/en-us/azure/devops/organizations/projects/public-projects-retirement?view=azure-devops
• https://docs.github.com/en/organizations/managing-user-access-to-your-organizations-repositories/managing-outside-collaborators/adding-outside-collaborators-to-repositories-in-your-organization
• https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization

Access levels — quick notes

• Access level = available web-portal features. Permissions = allowed actions and objects.
• Product owner or approver who does not need private-project Repos access → Stakeholder.
• Existing public projects temporarily give Stakeholders full Repos access; conversion to private projects is scheduled for 2027.
• GitHub contractor needing selected repositories without organization membership → outside collaborator; with Enterprise Managed Users, repository collaborator.
• Prefer Microsoft Entra group rules where practical, and review the Group rules tab regularly.

Primary sources: https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/az-400 · https://learn.microsoft.com/en-us/azure/devops/organizations/security/stakeholder-access?view=azure-devops · https://learn.microsoft.com/en-us/azure/devops/organizations/security/access-levels?view=azure-devops · https://learn.microsoft.com/en-us/azure/devops/organizations/billing/buy-basic-access-add-users?view=azure-devops · https://learn.microsoft.com/en-us/azure/devops/organizations/security/about-permissions?view=azure-devops · https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/assign-access-levels-by-group-membership?view=azure-devops · https://learn.microsoft.com/en-us/azure/devops/organizations/projects/public-projects-retirement?view=azure-devops · https://docs.github.com/en/organizations/managing-user-access-to-your-organizations-repositories/managing-outside-collaborators/adding-outside-collaborators-to-repositories-in-your-organization · https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization

Automate container scanning

A container image bundles your application and an operating system userland. Both carry vulnerabilities, and they age differently: your code changes when you change it, the base image accumulates CVEs on someone else's schedule.

Two things to scan

Where scanning goes in the pipeline

Scan before push so a vulnerable image never enters the registry, and scan in the registry continuously so an image that was clean at build time raises an alert when a new CVE lands. Both are needed for the same reason a quiet repository still needs Dependabot alerts: the code stopped changing, the threat landscape did not.

CodeQL in a container

Running CodeQL analysis inside a container is a documented objective, and it needs advanced setup — the generated workflow file is where you specify the container and the build. Default setup cannot express it.

The typical reason is a compiled language whose build environment lives in the container: CodeQL must observe the real build to analyse it, so the analysis has to run where that build runs.

Rebuilding is the fix

Most base-image findings are resolved by rebuilding on a patched base rather than by changing your code — which makes an automated periodic rebuild a security control, not merely hygiene.

Primary sources

• https://docs.github.com/en/code-security/code-scanning
• https://learn.microsoft.com/en-us/azure/container-registry/scan-images-defender

Container scanning — quick notes

• Scan before push → keep vulnerable images out of the registry.
• Scan in the registry continuously → catch CVEs disclosed after build.
• CodeQL in a container requires advanced setup — default setup cannot express it.
• Most base-image findings are fixed by rebuilding on a patched base.

Trap: scanning only at build time. The image ages even when your code does not.