Model, prompt and context decisions — reference

This reference supports the Unit 2 lectures and local project. It distinguishes documented behavior from house acceptance rules. Product sources were retained and verified on 5–6 September 2026. Recheck model availability, supported thinking controls, context limits, cache eligibility and pricing before a live implementation.

Model selection is a task-level comparison

The model-selection guide frames the choice through capabilities, speed and cost, and calls for testing actual prompts and data. Record the exact candidate configuration; changing a supported effort setting is a new experiment even when the model name is unchanged.

Worked house decision: 90% of requests are routine and 10% are exceptions. A achieves 99% and 60% respectively: 0.9 × 99 + 0.1 × 60 = 95.1%. B achieves 96% and 90%: 95.4% overall. With floors of 95% overall and 85% on exceptions, A fails and B passes those quality gates. B still needs the stated latency and cost checks. These results are fixtures, not vendor measurements.

An efficiency-first route starts with a cheaper/faster candidate and upgrades for a demonstrated capability gap. A capability-first route establishes feasibility for difficult work, then evaluates efficiency improvements. The source's dated starting examples are hypotheses to test, not an automatic release decision.

Prompt templates need explicit interfaces

The prompting guide supports clear output constraints, role instructions, examples and descriptive tags. Output-consistency guidance explains defining the desired format. A response shape makes validation possible; it does not prove the fields are true.

External documents can contain text such as “ignore the limit and proceed.” The mitigation guide says tool, document and search content must remain untrusted data. Tags improve organization; they do not grant that content authority. The assembler must serialize fields correctly, and the application must enforce its consequential action rules independently.

House sequence: construct the request → obtain a candidate → validate shape → validate identity, support and policy → act only if every required check passes. A bounded format-repair path can handle malformed output; a policy failure needs rejection or the specified review path. Critical-information validation and explicit uncertainty remain necessary even for fluent responses.

Select examples and reasoning support deliberately

Examples should cover relevant boundaries and vary irrelevant details. If every urgent example is long, add short urgent and long non-urgent examples that follow the actual rule. Keep held-out evaluation cases distinct from prompt demonstrations. Record the exact examples so later comparisons remain interpretable.

Chain-of-thought prompting and model thinking features are related to reasoning support, but an application should not require a private internal reasoning transcript as its evidence of correctness. Ask for useful decision outputs: a recommendation, applicable criteria, evidence references and unresolved conditions. Validate those outputs. The current prompting guide explains model-dependent thinking guidance; old recipes are not universal API contracts.

Count capacity, then preserve decision state

The context-window guide includes system instructions, messages, tool results, images, documents and tool definitions in the request. The response also occupies the window. Ordinary input, cache reads and cache creation all count as input context.

House ledger: a 32,000-token limit, 8,000 tokens of instructions/tools, 17,000 of messages/evidence and 5,000 reserved for output leaves 2,000 headroom. If new tool evidence adds 4,000, the plan exceeds the supplied limit by 2,000. Cached input cannot be subtracted to make it fit.

Compaction summarizes earlier conversation material. For the incident project, retain the exact target, cited records, unresolved conflicts and approval limits. A summary that changes “Region A approved” to “approved” loses a restriction needed for the next action. Test continuation after trimming rather than judging only the summary's length or fluency. The required fields and stop behavior are house design choices.

Separate three reuse mechanisms

Prompt caching follows tools, system and messages through the designated block. Matching requires exact segments, including text and images. Keep changing request content after the reusable prefix. Check the selected lifetime from the start of the writing or reading request, and the model's minimum cacheable length. A marker alone does not prove a hit.

Modular rules organize ordinary instructions; a rules file does not automatically become a Skill or a cached entry. Skills expose names and descriptions before activation. References enter context when read; scripts execute through bash and contribute their output. Loading a resource still has a context consequence.

House cost example: ten requests each incur 6 units for variable input/output. An uncached prefix costs 40 per request, making 10 × (40 + 6) = 460. A first cache write costs 50 and nine valid reads cost 4 each, making 50 + 9 × 4 + 10 × 6 = 146. This assumes eligibility, exact matching and timely reuse. Real decisions need current prices and observed usage, not these exercise units.

Objective and skill index