Operations and observability
Fewer tokens without losing the line that matters
Starting questionHow do we cut the cost of tool output in AI agents without handing customer data to external compression services and without losing important information?
Result
Open
Design in testing: a local filter that protects error lines and identifiers. It measures first, before it changes any output.
With a local, deterministic filter instead of an external service: error lines, paths, exit codes, and identifiers stay untouched; structured data passes through unchanged. The filter runs in measuring mode first to prove that nothing essential is lost.
What we tested
Until now, long output was simply truncated. The error message at the end of a test run got lost in the process.
The design calls for a local, deterministic filter. Error lines, paths, exit codes, and identifiers stay untouched; structured data passes through unchanged. The filter first runs in measurement mode without altering any responses.
What it means for business architecture
Optimizing AI costs is also a question of data sovereignty. External services are ruled out for customer data. The optimization has to happen in-house and prove that nothing essential is lost.
Lessons learned
- Truncating is not compressing. It often removes the most important part.
- A measurement mode before switching over builds trust in the savings.
- Adopt the method from a tool, not its code. The tool does not know your requirements for tenants and logging.
- tokens
- AI agents
- data sovereignty
- cost optimization