Protege

Appearance

Observing your agent ​

Sundae and Scout are working — but an agent you can't see into is an agent you can't trust. This chapter tours the two ways Protege lets you watch: the live introspection panel for what's happening right now, and tracing for a durable record you can review later.

The dashboard ​

Everything lives under the mount you set up — http://localhost:3000/protege. It's a real, if compact, console:

  • Inbox — every conversation with full message history, the compose form, and in-thread replies. The Search emails… link in its sidebar finds any message across threads by sender, recipients, subject, body, or attachment.
  • Agents — your agents (Sundae, Scout), their instructions, attached toolkits, and access rules.
  • Toolkits — the tool bundles themselves: membership checklists, and the system toolkits the engine maintains.
  • Responsibilities — Scout's duties and the history of each run.
  • Domains — registered email domains and their DNS records.
  • Traces — captured inference turns (when tracing is on; see below).

The whole console is keyboard-driven if you like it that way — press ? for the shortcut overlay (g 1–g 6 jump between sections, g a starts a new agent, g c a new thread). And it's yours to build on: host pages can wear the same layout and component helpers, so your own agent-adjacent admin screens read as part of this console — see Extending the dashboard.

Remember the mount ships no authentication — fine on localhost, but you must wrap it before production. That's the top of the security checklist in going to production.

The introspection panel ​

Open a thread and you'll see a live activity feed alongside it — the introspection panel. As Sundae handles a message it streams the model's output and each tool call as a card (the arguments she passed, the result she got), under a pulsing Processing… marker that clears when the run completes — or turns into an error marker when it fails. When a customer asks about order SCP-1042, you watch lookup_order fire with { "order_number": "SCP-1042" }, see the status come back, then see send_email compose the reply — in real time.

This is the fastest way to understand — or debug — an agent: you see exactly what it saw and did, in order. It's driven by the same events that power hooks, so it costs the run nothing.

The panel is live-only

The feed shows activity as it happens; it isn't persisted, so reloading the page starts it empty. For a durable record, use tracing.

Tracing ​

Where the panel is ephemeral, tracing captures a durable snapshot of every inference turn — the exact request sent to the model, the response it produced, and the settings that shaped it. It's off by default; turn it on in the initializer:

ruby
# config/initializers/protege.rb
config.tracing = { enabled: true }

Now every turn is saved as a trace, browsable under Traces, newest first. Each shows the correlation id (which ties all the turns of one run together), the model, and the full request/response JSON.

Reviewing and searching ​

Traces are built for two things:

  • Review. Open a trace and apply a label — good, bad, or neutral — with a note. Over time this builds a labeled dataset: the good turns are fine-tuning material, the bad ones show you where an agent went wrong.
  • Search. The trace search filters by correlation id (pull up every turn of one run — the workhorse when debugging a specific conversation), by model, by free text against the request or response JSON (e.g. every turn mentioning "refund"), or by review label.

See Tracing for the full field list and the current limitations (no export or auto-labeling yet).

Next ​

You can build agents, scope them, schedule them, and watch them. The last step is shipping The Scoop for real.

→ Going to production.