Building a real tool around the Claude Code CLI: streaming and guardrails
A Claude Code skill or agent is a markdown file with a YAML frontmatter. Nothing complicated when you have two or three. The problem starts once they pile up: fixing a skill by hand with no immediate feedback on frontmatter errors, and losing all trace of what was launched, when, with what result, and at what cost. These two frictions, not just one, are what I wanted to fix with Tarmac.
Two frictions, two answers#
| Friction | Answer |
|---|---|
| Fixing a skill by hand, with no immediate feedback | Editor with live frontmatter validation (name, description, allowed tools, model) |
| Losing track of what ran | A history for every run: duration, cost, tokens, status |
The second answer is the one that demanded the most architectural thinking, because it means actually running a process, not simulating a result.
No simulation: streaming a real CLI process#
The non-negotiable starting point: Tarmac isn't a wrapper displaying a prefabricated result. Every launch is a real call to the official claude CLI, in headless mode (claude -p), with the response text streamed live into the interface as it's produced by the process.
Technically, that means reading a subprocess's standard output as it flows and pushing it to the browser, rather than waiting for execution to finish to display everything at once. The difference isn't just cosmetic: on a skill that takes thirty seconds, watching the reasoning build live completely changes your ability to catch a problem before the run ends, instead of discovering the failure after the fact.
Cost and tokens aren't a footnote#
A skill run that "worked" says nothing about whether it cost three cents or three dollars. Without a history, that number disappears the moment the terminal closes. With a persistent history (local SQLite), the question changes shape: you're no longer just asking "does this skill work," but "is running this skill every day actually worth it." That's a question you never ask yourself until the numbers are in front of you.
Demo mode, an underrated design choice#
One detail that matters more than it looks: Tarmac exposes a demo mode, a fake dataset that lets you show the interface publicly without exposing your real .claude configuration. Separating the demo from actual use isn't just a privacy concern, it's what makes the tool showable to someone without risking exposing your personal prompts and costs.
Why real execution stays local#
Tarmac's showcase is deployed on Vercel, but actually running skills stays deliberately local (localhost:3700). Running the Claude Code CLI against a real .claude folder requires filesystem access and credentials that have no business being on a shared public server. The most important guardrail isn't in the code, it's in that scoping decision: never publicly deploy an execution capability that grants access to someone's configuration.
What this generalizes to#
Building a tool around an agentic CLI reveals a rule that goes beyond this one project: the value doesn't come from the model call itself, it comes from everything around it, immediate error feedback, cost traceability, a clean separation between demo and real execution. The model is the easy part. The fabric of guardrails around it is what makes the tool usable day to day rather than a demo you show once.
