Monitoring pipeline health
Monitor pipeline health
The syllabus names three areas without ranking them. Measure each with a defined report rather than a subjective proxy.
| Area | Defined measure and report use |
|---|---|
| Failure/pass rate | Successful or failed runs divided by total runs over a chosen period; outcome reports also expose failed-run count by pipeline |
| Duration | How long successful or partially successful runs take, trended by day and percentile |
| Flaky tests | Tests with differing outcomes despite unchanged source/execution environment; report count/rate and affected runs |
Flaky-test behavior
Azure DevOps Services system detection can rerun a failed test in the same pipeline execution; a fail-then-pass result can be marked flaky. Custom detection can also mark or unmark tests through the API. Flaky result data and tags support reports and troubleshooting. Suppressing flaky failures from the test summary or pipeline result is a configurable choice, not an automatic rule.
Run diagnosis versus trends
An individual run remains useful for diagnosing that execution. Trends add a different view: they reveal changes and recurring patterns in pass/fail rate, completion duration, test duration, failures, and non-deterministic outcomes. Compare like-for-like periods and investigate the underlying runs rather than treating a trend alone as a diagnosis.
Primary sources
- https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/az-400
- https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-pipelines-pass-rate-trend
- https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-pipelines-allpipelines
- https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-pipelines-duration-trend
- https://learn.microsoft.com/en-us/azure/devops/pipelines/test/flaky-test-management
- https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-test-analytics-flaky-tests
- https://learn.microsoft.com/en-us/azure/devops/report/dashboards/configure-test-results-trend