Integration project — stage 1: reduce capability bloat

Difficulty: intermediate architecture exercise. Time: 35–50 minutes. Cost: none; use the synthetic records below. Do not call paid APIs for this stage. Prerequisite: complete the tool-configuration lecture and distinguish tool search, caching and context editing.

Objective: CCARP-U3.T1.LO1. Produce a defensible configuration decision with an evaluation and rollback record. This is the first stage of the Integration project; later stages are not yet delivered.

Inputs — synthetic, not vendor benchmarks

A support assistant has 80 available tools. Two tools are typically needed for a task. Its startup request is dominated by unused definitions. The product owner requires at least 19 successful tasks out of the same 20-case fixture and a p95 response time no greater than 4.0 seconds. These are house exercise thresholds.

For this exercise, B repairs the missed-capability descriptions identified in A's failure traces. A separate long-session trace contains obsolete result blocks after the decision evidence has been recorded. Do not infer that these synthetic numbers predict performance on a real system.

Steps and required outputs

1. Create catalog-inventory.md with columns for capability, expected task, overlap, discovery description and side-effect boundary. Include six illustrative tools: ticket lookup, ticket update, account lookup, invoice lookup, knowledge search and incident escalation. Mark them as house examples. Add get_ticket as an alias of ticket lookup with identical inputs, outputs, permissions and backend behavior. Compare the two contracts and propose one canonical interface; do not infer equivalence from names alone.
2. Create decision.md. Identify the source of startup pressure, select the candidate that meets both stated acceptance thresholds and explain why the other candidates do not satisfy the same optimization goal.
3. Draw the discovery sequence: assistant requests capability → definition returned → selected tool invoked → result returned. Label the lookup exchange as part of end-to-end latency. Include the same sequence in words.
4. Create evaluation.md with the supplied summary statistics, the success-rate calculation (19/20 = 95%), the stated p95 comparison and rollback condition. Interpret the supplied p95 values; the exercise does not provide raw timings from which to recompute them. Record task success separately from catalog size. Explain why a cache hit does not reduce the occupied context in these inputs.
5. Create history-policy.md for the separate long-session trace. Name the evidence that must survive cleanup and explain why old-result editing solves a different problem from deferred definitions.

Evidence limit

This stage is an interpretation and design-record exercise using supplied aggregate results. It does not ask you to reproduce the vendor implementation or derive a p95 from absent raw timings. A production evaluation must retain per-case outcomes, timings and diagnostic traces.

Acceptance checks

• The selected candidate achieves at least 19/20 successful tasks and p95 ≤ 4.0 seconds under the supplied fixture.
• The decision addresses the measured startup cause and includes the discovery lookup in timing.
• Every tool in the proposed final catalog has a distinct task and a stated side-effect boundary; adding a tool is justified by a task need.
• The diagram and its text equivalent show the same order.
• The history policy preserves required decision evidence and distinguishes cost from context occupancy.
• Rollback has an owner, trigger and restoration step. No real customer data or credentials appear in the outputs.

Reference solution

Select pilot B for this synthetic fixture. It meets 19/20 and 3.9 seconds while reducing initial definition load. Pilot A fails the quality threshold despite using fewer tokens. The cached eager catalog meets the outcome thresholds but leaves the measured startup context pressure unchanged; it is a cost intervention, not the demonstrated context reduction. The baseline remains the rollback target.

A reasonable rollback trigger is fewer than 19 successful fixture tasks or p95 above 4.0 seconds after a configuration change. The named owner restores the prior catalog configuration, verifies the fixture again and records the failing capability trace before attempting another repair. Production criteria would require a representative sample and agreed statistical treatment; the small fixture here teaches the decision, not production assurance.

Use context editing for the separate obsolete-result problem after retaining needed evidence. It cannot replace the initial definition-loading decision. A tool search configuration is also not an authorization control: the executor must still enforce the operation's permissions. That last point is a house security design requirement to be developed in the next Integration stage.

Cleanup

Keep the five review artifacts as your submission. Delete only temporary synthetic trace files that you created. No cloud resources, API keys or customer records should have been created.

Source

Define tools — Claude documentation supports clear descriptions and consolidation of related operations.

Manage tool context — Claude documentation supports the distinctions among discovery, caching and context editing. All scenario numbers, tools, thresholds and review procedures are explicitly house-authored.