Lifecycle project — a handover pack somebody else could run

Build a local harness that produces the paperwork around a Claude-backed claims triage assistant: the requirements it was built to, the decision that chose its model, the service level it was promised at, the handover it ships with, and the loop that closes after a failure. The harness uses fixed fixtures. It tests your communication and lifecycle decisions; it does not call a model, host an agent, measure latency or send anything anywhere. Every threshold, latency figure, cost and owner name below is a house fixture.

Use Python 3's standard library. No credentials or paid services are required. Complete each stage before reading its expected output, then save and run your work as unit6_project.py.

Difficulty: advanced delivery practice. Estimated duration: 120-150 minutes. Prerequisites: the three Unit 6 topic lectures.

Stage 1 - Sort the intake into needs and decisions

Maps to CCARP-U6.T1.LO1.S1 and .S2.

Discovery produced five lines. Only some of them are requirements.

Line	Text
R1	The assistant should be highly accurate
R2	Use the fastest available model
R3	90% of replies begin within 2s, measured as time to first token
R4	Claims handlers must be able to see which policy clause was used
R5	Use Claude Haiku 4.5

Write classify(line) returning requirement, decision or wish, and is_criterion(line) requiring a bound, a share, a stated population and a named measurement. classify must read the text: a line that names a model is a decision whoever wrote it. Then write boundaries(line), recording the stakeholders each surviving requirement implies and the data it lets the assistant touch.

Expected output: R2 and R5 classify as decisions — each names a model, which is an answer to a question nobody wrote down, and neither can be missed. R1 is a wish: nothing could contradict it. R3 and R4 are requirements, and only R3 is already a criterion, because it carries a bound, a share, a population and a named measurement. Note that bound and share are separate fields on purpose: Stage 3 fails a target that has one and not the other, so they cannot both be called the threshold. Your note on R1 says what it has to acquire: a threshold on a named metric over a stated held-out set, chosen so it is achievable rather than aspirational. boundaries names a claims handler and a reviewer for R4, and marks R3 as touching no customer data at all — the requirement nobody thinks to write down is the one that says what the system may not see.

Stage 2 - Build a decision record that survives a challenge

Maps to CCARP-U6.T2.LO2.S1 and .S2.

Write record_gaps(record) over the fixture record, checking it carries a decision, the alternatives considered, and evidence from tests run on the real prompts and data. Then write business_line(choice), translating each technical option into the sentence a sponsor can agree to.

Option	Technical effect	Fixture monthly cost
Keep the smaller model	Meets the accuracy criterion today	$4,100
Raise effort within the model	More reasoning, more latency and cost	$6,300
Switch to a larger model	Higher capability, higher price	$18,500

Expected output: the fixture record fails on alternatives and on evidence. The alternatives list must include the effort setting that was not changed, not only the models that were not chosen — tuning effort is often a better lever than switching models. Each business line names what is gained and what is given up, and none of them says "better".

Stage 3 - Write a service level you could actually be held to

Maps to CCARP-U6.T2.LO3.S1, .S2 and .S3.

Write target_gaps(target) requiring a named metric, a threshold, and the share of requests it covers. Then write who_commits(platform) over the deployment options.

Platform	Fixture deployment
Claude API	Primary region
Microsoft Foundry	Secondary region
Amazon Bedrock	Regulated subsidiary

Expected output: the draft target "responses in under two seconds" is missing the metric and the share, because baseline latency and time to first token are different intervals. who_commits returns the first-party commitment for the Claude API, the Claude API schedule for Foundry, and Bedrock's own dates for the subsidiary — so the availability sentence in the subsidiary's contract cannot be copied from the others. Your pilot note revises the target from measured evidence, using an A/B comparison against the earlier version and task completion rates, rather than from the strength of anyone's opinion.

Stage 4 - Answer the pre-launch decisions in writing

Maps to CCARP-U6.T3.LO4.S1 and .S2.

Write handover_gaps(pack) over the six documented pre-launch decisions, and runbook(limitations) turning each known limitation into an operator action.

Pre-launch decision	Fixture answer
Session and state persistence	Store configured; memory files unaddressed
Observability	Exporters set
Auth and secrets	Inbound at the gateway; outbound unaddressed

Expected output: the pack has two gaps, and both are the same mistake — an arrangement that covers part of a category is recorded as covering the category. A session store mirrors transcripts, not memory files or other working-directory artifacts. Outbound tool credentials belong outside the agent environment, injected by a proxy after the request leaves the container. Your runbook binds the tool-use round trips because a session does not time out on its own, batches wide fan-outs, and records that the stall watchdog fires on absent output and is not a total-runtime deadline. It also alerts on the mirror error, with the sentence saying why: the run succeeds and the record is gone.

Stage 5 - Close the loop, and label the number honestly

Maps to CCARP-U6.T3.LO5.S1, .S2 and .S3.

Write unowned(phases) over the five lifecycle phases, next_case(failure) turning a monitored failure into an evaluation case, and report(runs) over the fixture run ledger. The report is an estimate, and the function has to say so.

Run	Outcome	Fixture cost
A	success	$0.42
B	failed midway	$0.19
C	crashed, fields zeroed	recover from prior result

Expected output: the monitoring phase comes back unowned, which is why the alert nobody configured has nobody to page. The new evaluation case is placed in a set that mirrors the real task distribution and is structured for automated grading, so it re-runs every release rather than being reviewed by hand once. The report totals all three runs: a conversation that fails midway still consumed tokens up to the point of failure, and the crashed run's totals are recovered from what arrived before the zeroed result. And the report is labelled an estimate, because the SDK's dollar fields are client-side estimates rather than authoritative billing data, and the documentation says not to bill end users or trigger financial decisions from them. Your constraints note carries what design already knew — serving infrastructure can shift and produce minor observable differences on a stable model ID, and tracing is in beta.

Reference solution

python

"""Unit 6 project reference solution. Standard library only; no network calls.

Every threshold, latency figure, cost and owner name is a house fixture.
"""

import json

# ------------------------------------------------------------------ Stage 1
# House fixture. "bound" and "share" are separate fields because Stage 3 fails a
# target that carries one without the other, so neither can be "the threshold".
MODEL_NAMES = ("claude haiku", "claude sonnet", "claude opus", "claude fable", "fastest available model")
INTAKE = [
    {"id": "R1", "text": "the assistant should be highly accurate",
     "bound": None, "share": None, "population": None, "measurement": None,
     "stakeholders": ["claims handler"], "reads": []},
    {"id": "R2", "text": "use the fastest available model",
     "bound": None, "share": None, "population": None, "measurement": None,
     "stakeholders": [], "reads": []},
    {"id": "R3", "text": "90% of replies begin within 2s, measured as time to first token",
     "bound": 2.0, "share": 0.90, "population": "all replies",
     "measurement": "time_to_first_token", "stakeholders": ["claims handler"], "reads": []},
    {"id": "R4", "text": "handlers can see which policy clause was used",
     "bound": None, "share": None, "population": "every reply", "measurement": "clause shown",
     "stakeholders": ["claims handler", "quality reviewer"], "reads": ["policy text"]},
    {"id": "R5", "text": "use Claude Haiku 4.5",
     "bound": None, "share": None, "population": None, "measurement": None,
     "stakeholders": [], "reads": []},
]


def is_criterion(line):
    """A criterion carries a bound, the share it covers, a population and a metric."""
    return all(line[part] is not None for part in ("bound", "share", "population", "measurement"))


def classify(line):
    """Read the line. A line that names a model is a decision, whoever wrote it."""
    lowered = line["text"].lower()
    if any(name in lowered for name in MODEL_NAMES):
        return "decision"          # an answer to a question nobody wrote down
    if line["population"] is None:
        return "wish"              # nothing could contradict it
    return "requirement"


def boundaries(line):
    """Who the requirement is for, and what it lets the assistant see."""
    return {"stakeholders": line["stakeholders"],
            "may_read": line["reads"],
            "may_not_read": "anything not listed; the boundary is the list, not the intent"}


stage1 = {
    "classified": {line["id"]: classify(line) for line in INTAKE},
    "criteria": [line["id"] for line in INTAKE if is_criterion(line)],
    "boundaries": {line["id"]: boundaries(line)
                   for line in INTAKE if classify(line) == "requirement"},
    "R1_needs": "a threshold on a named metric over a stated held-out set, "
                "chosen against a benchmark or prior result so it is achievable",
}

# ------------------------------------------------------------------ Stage 2
RECORD = {
    "decision": "keep the smaller model",
    "alternatives": [],
    "evidence": None,
}
OPTIONS = [
    {"choice": "keep the smaller model", "gain": "meets the accuracy criterion today",
     "give_up": "headroom if the criterion tightens", "monthly_usd": 4100},
    {"choice": "raise effort within the model", "gain": "more reasoning on hard claims",
     "give_up": "latency and cost, inside one model", "monthly_usd": 6300},
    {"choice": "switch to a larger model", "gain": "higher capability",
     "give_up": "the largest cost step on the table", "monthly_usd": 18500},
]


def record_gaps(record):
    gaps = []
    if not record.get("decision"):
        gaps.append("decision")
    if not record.get("alternatives"):
        gaps.append("alternatives")
    if not record.get("evidence"):
        gaps.append("evidence")
    return gaps


def business_line(option):
    return (f"{option['choice']}: gains {option['gain']}, gives up {option['give_up']}, "
            f"at ${option['monthly_usd']:,} per month (house fixture)")


stage2 = {
    "gaps": record_gaps(RECORD),
    "alternatives_must_include": "the effort setting that was not changed, "
                                 "not only the models that were not chosen",
    "evidence_must_be": "benchmark tests specific to the use case, run on the real prompts and data",
    "business_lines": [business_line(option) for option in OPTIONS],
}

# ------------------------------------------------------------------ Stage 3
DRAFT_TARGET = {"metric": None, "bound_seconds": 2.0, "share_of_requests": None}
LATENCY_METRICS = {
    "baseline_latency": "prompt processed and response generated",
    "time_to_first_token": "prompt sent to the first token of the response",
}
PLATFORMS = {
    "claude_api": "first-party commitment",
    "claude_platform_on_aws": "first-party commitment",
    "microsoft_foundry": "follows the Claude API lifecycle schedule",
    "amazon_bedrock": "sets its own lifecycle dates",
    "google_cloud": "sets its own lifecycle dates",
}


def target_gaps(target):
    gaps = []
    if target["metric"] not in LATENCY_METRICS:
        gaps.append("metric")      # two intervals exist; the number alone is ambiguous
    if target["bound_seconds"] is None:
        gaps.append("bound")
    if target["share_of_requests"] is None:
        gaps.append("share")       # a mean hides the tail the user actually feels
    return gaps


def who_commits(platform):
    return PLATFORMS[platform]


stage3 = {
    "draft_gaps": target_gaps(DRAFT_TARGET),
    "repaired": target_gaps({"metric": "time_to_first_token",
                             "bound_seconds": 2.0, "share_of_requests": 0.90}),
    "commitments": {name: who_commits(name)
                    for name in ("claude_api", "microsoft_foundry", "amazon_bedrock")},
    "subsidiary_note": "the availability sentence cannot be copied from the primary region",
    "renegotiate_from": ["A/B comparison against the earlier version", "task completion rates"],
}

# ------------------------------------------------------------------ Stage 4
PRE_LAUNCH = ("session_and_state", "observability", "auth_and_secrets",
              "scaling_and_concurrency", "cost", "multi_tenant_isolation")
PACK = {
    "session_and_state": {"answered": True, "covers": ["transcripts"],
                          "category": ["transcripts", "memory_files"]},
    "observability": {"answered": True, "covers": ["traces", "metrics", "logs"],
                      "category": ["traces", "metrics", "logs"]},
    "auth_and_secrets": {"answered": True, "covers": ["inbound"],
                         "category": ["inbound", "outbound"]},
    "scaling_and_concurrency": {"answered": True, "covers": ["measured ceiling"],
                                "category": ["measured ceiling"]},
    "cost": {"answered": True, "covers": ["token accounting"], "category": ["token accounting"]},
    "multi_tenant_isolation": {"answered": True, "covers": ["per-tenant dirs"],
                               "category": ["per-tenant dirs"]},
}
LIMITATIONS = [
    ("no_session_timeout", "bound the tool-use round trips with a maximum turn count"),
    ("memory_growth", "cap session length or recycle subprocesses periodically"),
    ("wide_fanout_rate_limits", "break the work into smaller batches"),
    ("no_subagent_deadline", "the stall watchdog fires on absent output; it is not a runtime deadline"),
]


def handover_gaps(pack):
    """A decision that covers part of a category is not an answered decision."""
    gaps = []
    for decision in PRE_LAUNCH:
        entry = pack.get(decision)
        if not entry or not entry["answered"]:
            gaps.append((decision, "unanswered"))
            continue
        missing = [part for part in entry["category"] if part not in entry["covers"]]
        if missing:
            gaps.append((decision, "partial: " + ", ".join(missing)))
    return gaps


def runbook(limitations):
    entries = {name: action for name, action in limitations}
    entries["mirror_error"] = ("alert: the batch is dropped and the query continues, "
                               "so the run succeeds and the record is gone")
    return entries


stage4 = {"gaps": handover_gaps(PACK), "runbook": runbook(LIMITATIONS)}

# ------------------------------------------------------------------ Stage 5
PHASES = [
    {"phase": "discovery", "owner": "product"},
    {"phase": "design", "owner": "architecture"},
    {"phase": "handoff", "owner": "architecture"},
    {"phase": "monitoring", "owner": None},
    {"phase": "iteration", "owner": "engineering"},
]
FAILURE = {"id": "F-118", "symptom": "cited a lapsed policy clause", "gradable": "string match"}
RUNS = [
    {"run": "A", "outcome": "success", "total_cost_usd": 0.42, "fields_zeroed": False},
    {"run": "B", "outcome": "error", "total_cost_usd": 0.19, "fields_zeroed": False},
    {"run": "C", "outcome": "error_during_execution", "total_cost_usd": 0.0,
     "fields_zeroed": True, "prior_result_usd": 0.33},
]


def unowned(phases):
    return [entry["phase"] for entry in phases if not entry["owner"]]


def next_case(failure):
    return {
        "case_id": failure["id"],
        "placed_in": "an eval set that mirrors the real-world task distribution",
        "graded_by": failure["gradable"],   # automated, so it re-runs every release
        "not": "a single case reviewed by hand before release",
    }


def report(runs):
    """Failed conversations still spent tokens; a crashed result may be zeroed.

    The label is part of the return value on purpose. These dollar figures are
    client-side estimates, not authoritative billing data, so a report that does
    not say so invites somebody to bill a customer from it.
    """
    total = 0.0
    for run in runs:
        total += run["prior_result_usd"] if run["fields_zeroed"] else run["total_cost_usd"]
    return {"estimate_usd": round(total, 2),
            "basis": "client-side estimate for development insight and approximate budgeting",
            "authoritative_source": "the Usage and Cost API, or the Console usage page",
            "not_for": "billing end users or triggering financial decisions"}


stage5 = {
    "unowned_phases": unowned(PHASES),
    "next_case": next_case(FAILURE),
    "report": report(RUNS),
    "counted_runs": len(RUNS),
    "constraints_carried": [
        "serving infrastructure can change and occasionally produces minor observable "
        "differences on a stable model ID",
        "tracing is in beta; span names and attributes may change between releases",
    ],
}

# ------------------------------------------------------------- Stage checks
# Each assertion restates the expected output printed under its stage above, so
# running the file confirms the paperwork came out the shape the stage claimed.
CHECKS = [
    ("stage 1 sorts decisions out of the requirements",
     stage1["classified"] == {"R1": "wish", "R2": "decision", "R3": "requirement",
                              "R4": "requirement", "R5": "decision"}),
    ("stage 1 finds one criterion already complete", stage1["criteria"] == ["R3"]),
    ("stage 1 records who each requirement is for and what it may read",
     sorted(stage1["boundaries"]) == ["R3", "R4"]
     and stage1["boundaries"]["R4"]["may_read"] == ["policy text"]
     and stage1["boundaries"]["R3"]["may_read"] == []),
    ("stage 2 finds the record short of alternatives and evidence",
     stage2["gaps"] == ["alternatives", "evidence"]),
    ("stage 3 finds the draft target missing its metric and its share",
     stage3["draft_gaps"] == ["metric", "share"] and stage3["repaired"] == []),
    ("stage 3 sends the subsidiary to the platform's own dates",
     stage3["commitments"]["amazon_bedrock"] == "sets its own lifecycle dates"),
    ("stage 4 finds two partly-answered decisions",
     [name for name, _ in stage4["gaps"]] == ["session_and_state", "auth_and_secrets"]),
    ("stage 4 alerts on the mirror error", "mirror_error" in stage4["runbook"]),
    ("stage 5 finds the unowned phase", stage5["unowned_phases"] == ["monitoring"]),
    ("stage 5 counts every run, including the failure and the crash",
     stage5["counted_runs"] == 3 and stage5["report"]["estimate_usd"] == 0.94),
    ("stage 5 labels the figure an estimate rather than a bill",
     "estimate" in stage5["report"]["basis"] and stage5["report"]["not_for"]
     == "billing end users or triggering financial decisions"),
]

if __name__ == "__main__":
    print(json.dumps({"stage1": stage1, "stage2": stage2, "stage3": stage3,
                      "stage4": stage4, "stage5": stage5}, indent=2, default=str))
    failed = [name for name, passed in CHECKS if not passed]
    if failed:
        raise SystemExit("failed: " + "; ".join(failed))
    print("all stage checks passed")

What this project does not establish

The harness checks the shape of your paperwork, not its truth. It cannot tell you whether a threshold is the right one for your business, whether a latency bound is reachable on your traffic, or whether a contract clause is enforceable where you operate. Those are questions for measurement and for advisers.

Nothing here asserts that a documented commitment applies to your deployment. Check the platform your deployment actually reaches the model through, and read the current published dates, because every model ID has its own distinct deprecation and retirement schedule and the published dates are floors rather than appointments.