Engineering a production agent harness with Pi

Operating context

Production agent harness responds to operational events from collaboration, source-control, and platform systems. The agent needs enough context to investigate a task, but it also needs boundaries around tools, commands, cost, state, and delivery. I worked on the harness that supplies those boundaries around the model.

Pi runtime

I evolved the TypeScript runtime built around the Pi SDK and pi-agent-core. Declarative runbooks become system context, while deterministic prefetch gathers known evidence before the agent begins reasoning. LiteLLM routing lets the harness choose an appropriate model without coupling the workflow to one provider.

The harness exposes controlled tools for investigation and typed side effects. Command policy, execution budgets, and output limits constrain what a turn can do. Persistent sessions in Google Cloud Storage preserve the state required to resume work, and recovery logic re-engages turns whose final delivery was interrupted.

Evaluation and loop engineering

OpenTelemetry and Langfuse tracing connect model decisions, tool calls, and delivery outcomes. BigQuery feedback links user response to the run that produced it. Model comparison, judge calibration, annotation workflows, and instrumentation checks turn those traces into evidence for improving models, prompts, tools, and policy.

This creates connected loops around the agent: the model chooses and observes tools; the harness governs execution; product feedback records outcomes; and evaluation guides the next controlled change.

My contribution

My repository-backed work includes the Pi migration, runbook loading, persistent execution and recovery, tracing, feedback integration, model evaluation, judge calibration, and instrumentation contracts. These are my contributions within a system built and operated by a wider engineering team.

Team credit

The agent harness is a system built and operated by an engineering team. I describe my verified contributions while crediting the wider team that built and operates the platform.