OpsDesk AcademyGet the project Code · prompts · workflows
PART 3 OF 6 · ONE HOUR

Connect models. Know the cost.

Use your own model without guessing its capabilities.

YOUR DELIVERABLEOne real model run, a capability record and a cost estimate with explicit assumptions.
EXTEND THE SAME APP

Start with: The same app with the bounded engine from Part 2.

Add now: Model selection, capability checks and transparent token/cost reporting.

Download Part 3 agent instructions (.md)
SAME APP · CHOOSE YOUR WORKFLOW

Choose the depth that fits you.

All three paths build the same capabilities. The explanation and implementation method change; the required app result does not.

No IT knowledge / C-level

Browser-first. Describe the outcome, use a builder and validate the result in plain language.

Explain the words in this route
Backend
The private part of the app that calls AI and runs approved lookups; your builder can generate it.
Tool evidence
Information returned by an actual permitted lookup, not a model’s guess.
Guardrail
A rule enforced by the app before an action can run.

Bring: An owned builder account, available build credits and an explicitly selected app AI/backend option. No terminal is required for this route.

YOUR NEXT CAPABILITYModel selection, capability checks and transparent token/cost reporting.
  1. Keep the same app. Choose only explicitly supported models in your own backend/provider settings.
  2. Ask the builder to add a model selector backed by a server-owned allowlist; never a free-text endpoint field.
  3. Run the same synthetic investigation once per available model and compare correctness, time and reported usage.
  4. Display estimated cost only with matching sourced rates. Leave missing values unknown.

Browser-first: no terminal commands are required in this route. Use supported builder backend features; access and credits remain prerequisites.

Prompt for your builder or coding assistant
Extend our existing OpsDesk app for Part 3: Model selection, capability checks and transparent token/cost reporting. Follow this No IT knowledge / C-level learner route. Explain the next action in plain language and use the builder UI; do not require me to run terminal commands. Implement backend checks in code, not just in a prompt. Use synthetic data and my explicitly selected backend. Never invent a successful answer, usage or trace. Keep keys in server secrets. Do not publish, buy credits, activate cloud services or change permissions without specific authority. Complete the route actions, verify actual results and report blockers honestly.
Check your route steps

0 of 4 steps checked · self-reported

Download this persona’s Part 3 instructions (.md)

Go deeper · what each link explains
SEE THE RESULT · PART 3

Measure the answer, then compare models

Make latency, usage and estimated cost understandable without inventing a price or a winning model.

Try this

Use the same synthetic task for each explicitly selected, available model. Keep tool inputs and limits fixed.

  1. Read provider-reported input and output usage from the real run.
  2. Add matching sourced token rates only when available; otherwise show unknown.
  3. If a second model is available, compare answer quality, tool behavior, latency and usage on the same task.

What it should look like

A compact measurement panel beside the answer. Comparison rows distinguish reported tokens, illustrative arithmetic, estimated model cost and actual provider charges.

What a good result says

The reviewed Lovable run reported 761 input + 191 output = 952 total tokens and 5,218 ms. Estimated cost remained unknown. These are one run's observations, not a benchmark or a bill.

Observed checkpoint

Single-model usage browser-verified; cost arithmetic and missing-usage contracts locally tested.

Still required

No live two-model comparison or provider billing readback is accepted for this Lovable rehearsal.

App walkthrough and all expected results → · Reviewed evidence →

EXPLORE THE IDEA · SYNTHETIC ILLUSTRATION

No model call, permission decision, cloud action or telemetry export is performed by this illustration. Use your actual app and receipts for acceptance.
Shared engineering tasks · optional reference for the business route
00–15 MINUTES

1. Bind one actual model

A compatible URL does not prove native tool support.

  1. Copy the example configuration to a private path and set mode 0600.
  2. Select an already available model; set its exact alias and approved endpoint.
  3. For a gateway, reference its key by environment name in a private secrets file.
mkdir -p local/private/course-model
cp workshops/local-ai-studio/studio.example.toml local/private/course-model/studio.toml
chmod 600 local/private/course-model/studio.toml
EXPECTED RESULTConfiguration names one explicit model, allowed host and output bound.
If you get stuck

Do not print the secrets file or commit it. Provider accounts and weights are your own prerequisites.

15–30 MINUTES

2. Prove one native tool round-trip

A useful agent must read real handler output, not invent it.

  1. After editing the private configuration, start the runtime with a fresh output path.
  2. For a remote gateway, supply the key through the configured environment variable, or add --secrets with your own private 0600 env file. The local Ollama command needs no API key file.
  3. Open the local UI; ask a greeting, then ask for synthetic payments health.
  4. Record the returned model ID, actual tool name and outcome.
PYTHONPATH=src .venv/bin/python -m agent_workshop.studio --config local/private/course-model/studio.toml --output local/private/course-model/run-01 --port 8770
EXPECTED RESULTThe second task has an actual native tool call and observation; errors remain visible.
If you get stuck

A model that can chat but cannot issue native tools is not suitable for this lane. Stop and select an explicitly configured compatible model.

30–45 MINUTES

3. Compare behavior fairly

Change one variable when comparing models.

  1. Optionally add a second configured model alias using the same settings rules.
  2. Run the same synthetic task once per model; no automatic retries.
  3. Compare tool correctness, outcome, reported tokens and elapsed time.
Prompt for your coding assistant
Create a comparison worksheet for the same synthetic investigation across my configured models. Include actual model ID, tool-call validity, outcome, input/output tokens, latency and rate source. Leave unmeasured cells unknown. Do not run inference, choose a paid fallback or invent results.
EXPECTED RESULTYour comparison labels the exact task, returned model and known limitations.
If you get stuck

With only one model, complete a single-model baseline. Do not pretend it is a comparison.

45–60 MINUTES

4. Calculate only what is known

Reported token usage and billing are different evidence.

  1. Supply rates only when they match the returned model and have a source/date.
  2. Calculate (input tokens × input rate + output tokens × output rate) / 1,000,000.
  3. If either usage or rate is unknown, label the estimate unavailable.
PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_model_comparison.py' -v
EXPECTED RESULTA reviewer can reproduce the estimate; local GPU costs are not called zero.
If you get stuck

A provider may include cache or reasoning charges that this simple estimate does not model. Preserve that limitation.

COMPLETE THIS PART ONLY WHEN

Good questions. Short answers.

Does a model that chats necessarily support tools?

No. Verify an actual native tool-call round-trip with your selected alias. If it cannot issue the required calls, choose another explicitly configured model; keep that compatibility failure visible.

Check the runtime contract →
Why does my cost say unknown?

A defensible estimate needs reported usage and matching, sourced rates. Missing values are not zero. Local GPU electricity and provider invoices are different measurements from this token estimate.

Understand usage evidence →
Should I pick whichever model is fastest?

Compare the same synthetic task and consider correct tool use, grounded conclusions, outcome, usage and latency together. One quick answer is not a quality or reliability benchmark.

Record a comparable baseline →

Explain it back

Can missing usage be recorded as zero tokens?

Check the explanation

No. Missing measurement is unknown; zero is an observed value with different meaning.

No answer is uploaded or graded. Your Markdown download contains the full agent instructions plus your optional notes; review it before sharing.

0 tasks checked · self-reported, reset on reload

Primary references

Continue to Part 4 →