Governance project — a safety case you can hand to a reviewer

Build a local decision harness for a customer-support agent that can read tickets, comment on them, issue refunds and close accounts. The harness uses fixed fixtures. It tests your governance decisions; it does not call a model, run a tool, evaluate anything or touch a compliance system. Every threshold, score and policy rule below is a house fixture.

Use Python 3's standard library. No credentials or paid services are required. Complete each stage before reading its expected output, then save and run your work as unit5_project.py.

Difficulty: advanced governance design. Estimated duration: 120-150 minutes. Prerequisites: the two Unit 5 topic lectures.

Stage 1 - Place every control in the right row

Maps to CCARP-U5.T1.LO1.S1 and .S2.

Three questions need three different controls, and only the third can stop an action.

Row	Question it answers
Schema	Are the arguments well-formed?
Authorisation	May this caller do this?
Semantic safety	Should this happen now?

Write rows_covered(design) and empty_rows(design). Classify each control in the fixture design, then run the same functions over a design that has strict mode and nothing else. Record why a system-prompt policy is not one of the three rows.

Expected output: the full design leaves no row empty. The strict-mode-only design leaves authorisation and semantic safety empty, which is the shape of the failure this objective tests. Your note says a prompt-level policy is defence in depth because retrieved text can argue with it.

Stage 2 - Classify three failures that look identical

Maps to CCARP-U5.T1.LO2.S1 and .S2.

Three confident wrong answers are supplied. Write classify(failure) returning the underlying fault and its first remedy.

Case	Fact in evidence	Quote supports	Stable across runs
F1	No	No	Yes
F2	Yes	No	Yes
F3	Yes	Yes	No

Then record the abstention path and the residual risk that survives every fix.

Expected output: knowledge from outside, unsupported claim, and instability, each with a different remedy. The abstention path gives explicit permission to admit uncertainty and to say when no relevant quotes were found. The residual-risk line states that the mitigations reduce hallucination without eliminating it, so the irreversible action keeps a check that does not depend on the model.

Stage 3 - Gate on consequence and reversibility

Maps to CCARP-U5.T1.LO3.S1 and .S2.

Write gate(action) over the four fixture actions, and review_packet(action) for anything that needs approval.

Action	Harm	Reversible
Read a ticket	Low	Yes
Post an internal comment	High	Yes
Issue a refund	High	No
Delete an account	High	No

Expected output: run automatically, log and alert, require approval, require approval. Every approval packet carries the evidence the recommendation rests on and an approve-or-reject action, with nothing applied before approval. You also record the verification pass that runs before findings are surfaced, so reviewers are not drowned.

Stage 4 - Split the responsibility line

Maps to CCARP-U5.T2.LO4.S1 and .S2.

Name what the platform provides and what your deployment still owns. Record which terms govern each access path, both residency controls, the endpoint choice they imply, and the date from which audit history exists.

Expected output: both columns are filled. Direct commercial access is governed by the provider's terms with the addendum incorporated; the third-party marketplace path is governed by that platform's terms. Data residency and inference residency are named separately and a regional endpoint is chosen because both must hold. Audit history begins at enablement, and the earlier gap cannot be recovered. Your claim line says the platform supports compliance; your not-claimed line says this deployment is not thereby compliant.

Stage 5 - Write a fairness criterion that can fail

Maps to CCARP-U5.T2.LO5.S1 and .S2.

Write is_criterion(entry) requiring a threshold, a population and a named instrument. Then score the supplied slices against it.

Slice	Score
General	0.962
Assisted technology	0.947
Non-native speakers	0.883

Decide the release, publish the exclusions, and name the appeal path.

Expected output: the criterion is checkable. The aggregate passes the 0.92 threshold and the release fails, because one slice does not — which is the entire reason for scoring per slice. Your published exclusions say the mitigations reduce error without eliminating it, that auditability comes from asking for quotes rather than by default, and that no claim of verified freedom from bias is made.

Reference solution

python

"""Unit 5 project reference solution. Standard library only; no network calls."""

import json

# ------------------------------------------------------------------ Stage 1
# Three questions, three controls. Only the third can stop an action.
ROWS = ("schema", "authorization", "semantic_safety")
DESIGN = {
    "strict_tool_use": "schema",
    "tenant_and_role_check": "authorization",
    "consequence_gate": "semantic_safety",
    "system_prompt_policy": "defence_in_depth",   # not a row; it can be argued with
}


def rows_covered(design):
    return {row: sorted(k for k, v in design.items() if v == row) for row in ROWS}


def empty_rows(design):
    return sorted(row for row, controls in rows_covered(design).items() if not controls)


stage1 = {
    "rows": rows_covered(DESIGN),
    "empty_rows": empty_rows(DESIGN),
    "prompt_policy_is_not_a_row": DESIGN["system_prompt_policy"] == "defence_in_depth",
    "why": "retrieved text can argue with a check that lives inside the prompt",
}

# A design that trusts strict mode alone leaves two rows empty.
NAIVE = {"strict_tool_use": "schema"}
stage1["naive_design_empty_rows"] = empty_rows(NAIVE)

# ------------------------------------------------------------------ Stage 2
# Three failures that look identical from outside: a confident wrong answer.
FAILURES = [
    {"id": "F1", "fact_in_evidence": False, "quote_supports": False, "stable_across_runs": True},
    {"id": "F2", "fact_in_evidence": True, "quote_supports": False, "stable_across_runs": True},
    {"id": "F3", "fact_in_evidence": True, "quote_supports": True, "stable_across_runs": False},
]


def classify(failure):
    if not failure["fact_in_evidence"]:
        return ("knowledge_from_outside",
                "instruct it to use only the provided documents, not its general knowledge")
    if not failure["quote_supports"]:
        return ("unsupported_claim",
                "require a supporting quote per claim, and retract where none is found")
    if not failure["stable_across_runs"]:
        return ("instability",
                "repeat the prompt and compare; inconsistency can itself indicate hallucination")
    return ("unclassified", "probe further before changing any component")


stage2 = {
    "classified": {f["id"]: classify(f) for f in FAILURES},
    "abstention_path": "explicit permission to admit uncertainty, and to state when no "
                       "relevant quotes were found",
    "residual_risk": "the mitigations reduce hallucination without eliminating it, so the "
                     "irreversible action keeps a check that does not depend on the model",
}

# ------------------------------------------------------------------ Stage 3
ACTIONS = [
    {"id": "A1", "name": "read a ticket", "harm": "low", "reversible": True},
    {"id": "A2", "name": "post an internal comment", "harm": "high", "reversible": True},
    {"id": "A3", "name": "issue a refund", "harm": "high", "reversible": False},
    {"id": "A4", "name": "delete an account", "harm": "high", "reversible": False},
]


def gate(action):
    if action["harm"] == "low" and action["reversible"]:
        return "run_automatically"
    if action["reversible"]:
        return "log_and_alert"
    return "require_approval"


def review_packet(action):
    """A gate with no evidence is a rubber stamp; evidence with no action is a reading
    exercise. Both, or the gate is ceremonial."""
    return {"evidence": "quotes and sources for each claim the recommendation rests on",
            "action": f"approve or reject: {action['name']}",
            "applied_before_approval": False}


stage3 = {
    "gates": {a["id"]: gate(a) for a in ACTIONS},
    "packets": {a["id"]: review_packet(a) for a in ACTIONS if gate(a) == "require_approval"},
    "verify_before_surfacing": "an adversarial pass over each finding, so reviewers are not "
                               "drowned in false positives",
}

# ------------------------------------------------------------------ Stage 4
PLATFORM_SIDE = [
    "support for data subject rights, retention policies and deletion capabilities",
    "customer data from commercial deployments not used to train models by default",
    "a Data Processing Addendum defining roles and responsibilities",
    "certifications covering the platform",
]
CUSTOMER_SIDE = [
    "responding to data subject requests within the statutory deadline",
    "the organisation's own retention policy and its enforcement",
    "evidence about this deployment, for this auditor",
    "workforce training and access review",
]
ACCESS_PATHS = {"direct_commercial": "provider commercial terms, addendum incorporated",
                "third_party_marketplace": "that platform's terms of service"}
RESIDENCY = {"data": "where prompts, outputs and history are stored",
             "inference": "where requests are processed and responses generated"}

stage4 = {
    "platform_side": PLATFORM_SIDE,
    "customer_side": CUSTOMER_SIDE,
    "both_sides_named": bool(PLATFORM_SIDE) and bool(CUSTOMER_SIDE),
    "access_paths": ACCESS_PATHS,
    "residency_controls": RESIDENCY,
    "endpoint": "regional, because both residencies must be satisfied together",
    "audit_history_available_from": "the date the activity feed was enabled; earlier "
                                    "activity is not backfilled and cannot be recovered",
    "claim": "the platform supports compliance with industry and regional standards",
    "claim_not_made": "that this deployment is thereby compliant",
}

# ------------------------------------------------------------------ Stage 5
SLICES = {"general": 0.962, "assisted_technology": 0.947, "non_native_speakers": 0.883}
FAIRNESS = {"threshold": 0.92, "population": "1,000 tickets per slice",
            "instrument": "the same code-graded routing check used for task fidelity"}


def is_criterion(entry):
    return isinstance(entry.get("threshold"), (int, float)) and bool(entry.get("population")) \
        and bool(entry.get("instrument"))


def slice_verdict(scores, threshold):
    return {name: score >= threshold for name, score in scores.items()}


verdicts = slice_verdict(SLICES, FAIRNESS["threshold"])
stage5 = {
    "criterion": FAIRNESS,
    "is_checkable": is_criterion(FAIRNESS),
    "per_slice": verdicts,
    "aggregate": round(sum(SLICES.values()) / len(SLICES), 4),
    "aggregate_would_pass": round(sum(SLICES.values()) / len(SLICES), 4) >= FAIRNESS["threshold"],
    "release": all(verdicts.values()),
    "published_exclusions": [
        "the mitigations reduce error without eliminating it",
        "responses are auditable when quotes and sources are requested, not by default",
        "no claim that the system is verified free of bias",
    ],
    "appeal_path": "a named route for a user to contest an outcome, and a correction record",
}

# ------------------------------------------------------------------ Checks
assert stage1["empty_rows"] == []
assert stage1["naive_design_empty_rows"] == ["authorization", "semantic_safety"]
assert stage1["prompt_policy_is_not_a_row"]
assert [stage2["classified"][k][0] for k in ("F1", "F2", "F3")] == \
    ["knowledge_from_outside", "unsupported_claim", "instability"]
assert stage3["gates"] == {"A1": "run_automatically", "A2": "log_and_alert",
                           "A3": "require_approval", "A4": "require_approval"}
assert all(not p["applied_before_approval"] for p in stage3["packets"].values())
assert stage4["both_sides_named"] and len(PLATFORM_SIDE) >= 3 and len(CUSTOMER_SIDE) >= 3
assert set(stage4["access_paths"]) == {"direct_commercial", "third_party_marketplace"}
assert "that platform's terms" in stage4["access_paths"]["third_party_marketplace"]
assert set(stage4["residency_controls"]) == {"data", "inference"}
assert stage4["endpoint"].startswith("regional")
assert "not backfilled" in stage4["audit_history_available_from"]
assert stage4["claim"].startswith("the platform supports compliance")
assert "compliant" in stage4["claim_not_made"]
assert stage5["is_checkable"] and not is_criterion({"threshold": 0.92, "population": "x"})
# The aggregate passes and the release does not: that is the whole point.
assert stage5["aggregate_would_pass"] and not stage5["release"]
assert verdicts["non_native_speakers"] is False

print(json.dumps({"stage1": stage1, "stage2": stage2, "stage3": stage3,
                  "stage4": stage4, "stage5": stage5}, indent=2))
print("\nall stage checks passed")

Acceptance checklist

The strict-mode-only design reports two empty rows.
Stage 2 returns three different fault classes with three different remedies.
Every approval packet carries both evidence and an action, and applies nothing first.
Stage 4 fills the customer column, not just the platform column.
Stage 5's aggregate passes while the release fails.
The exclusions list contains no claim of verified fairness.

Explain the result before extending it

Three stages have an answer that is easy to produce and hard to justify.

Stage 1. The strict-mode-only design is not a straw man. It is what a team builds when a passing check feels like a safe system, and the two empty rows are invisible precisely because the one that is filled works perfectly.
Stage 3. The middle row is the one people argue about. Logging and alerting on a high-harm reversible action is a real decision with a real cost: someone has to read the alert in time. Say who.
Stage 5. The aggregate is not wrong; it is answering a different question. Reporting it alone is how a system ships while failing the people least able to complain about it.

Cleanup

Nothing to clean up: no credentials, no network calls, no files written outside your working directory. Delete unit5_project.py when you are done, or keep it as the starting point for the Unit 6 project.

What this project does not establish

These fixtures test your decisions and interfaces. They do not measure a real model, a real deployment or a real compliance posture, and nothing here is legal advice. Passing every stage means your governance records the right decisions with the right evidence, which is what this domain's objectives assess. Whether a given deployment satisfies a given regulation is a question for the organisation and its advisers.