Agent-Native Format
A simple question about a file type for AI agents, tested as a wider study of how agents read, retrieve, cache, and spend context.
I started Agent-Native Format with a simple question: could a file type made for AI agents help them read context faster, waste less work, and use fewer credits?
The situation.
My first thought was to invent a new file extension with a compact syntax. The more I explored it, the less interesting the extension became. The real question is how information should be structured and how much work an agent has to do before it can find and use the right part.
Formats such as TOON already explore representing structured data with fewer tokens. Agent context is a wider problem. An agent may need repository instructions, architecture documents, file maps, previous decisions, task state, dependencies, constraints, tool output, and relationships spread across all of them.
A 30% smaller file would be useful, but the bigger win might be helping an agent avoid reading 90% of the file. That points towards indexed context, useful blocks, stable cacheable sections, and selective loading rather than compression alone.
What I did.
I turned the idea into a public TypeScript research project and built Benchmark v0 before designing a final format. It starts with one canonical agent-context dataset containing project metadata, files, dependencies, decisions, people, constraints, issues, and milestones. That same information is represented as Markdown, JSON, YAML, minified JSON, and TOON 4.1.1.
I created nine machine-graded tasks covering exact lookup, multi-record filtering, cross-reference reasoning, constraint checks, repeated reads, and selective retrieval. The runner supports exact, set, and rubric scoring, seeded format ordering, repeated runs, raw JSONL records, manifests, and JSON and CSV summaries.
I also documented the limits of the work. Format order must be randomised, model versions and pricing must be recorded, failures cannot disappear from the results, and provider caching must be reported separately. The goal is to stop a smaller token count from being presented as a win when accuracy or reliability has fallen.
How the work unfolded.
Keep the facts identical
Generate or validate every representation against one canonical fixture so the benchmark compares structure rather than quietly changing the information available to the agent.
Test realistic context tasks
Ask the model to find exact values, filter records, follow references, check constraints, revisit earlier context, and retrieve a small relevant subset from distracting information.
Measure successful work
Record input and output tokens, accuracy, latency, repeated-read efficiency, selective-retrieval efficiency, estimated cost, and cost per successful task. Accuracy remains the first condition for calling something better.
Make results auditable
Preserve prompts, responses, scorer decisions, fixture hashes, model configuration, retries, failures, timestamps, and pricing snapshots instead of publishing only a favourable aggregate.
Learn what sits below the agent layer
Use the project to understand tokenisation, context windows, prompt caching, retrieval, indexing, chunking, model behaviour, latency, and API pricing more deeply than normal prompt and tool use requires.
What happened.
Benchmark v0 now represents the same agent context in Markdown, JSON, YAML, minified JSON, and TOON, then tests each representation across nine machine-graded tasks. The runner validates fixtures, randomises format order, scores responses, repeats trials, and reports accuracy, token use, latency, and cost per successful task.
Building the harness changed the project from an idea about a new file extension into a measurable question about context structure. It also made the likely direction broader: the useful result may be a compiled, indexed, and selectively loaded context container rather than one compact syntax.
What I learned.
The extension is the least interesting part
A memorable suffix does not help if the agent still has to scan everything, reconstruct relationships, or repeatedly load unchanged context.
Fewer tokens can still cost more
A compact representation is not an improvement if it produces more failures, longer answers, extra retries, or weaker comprehension. Cost per successful task captures that trade-off better.
Caching and retrieval need separate tests
Provider caching, selective loading, and the representation itself can each reduce work. The benchmark has to separate them before attributing an improvement to the format.
A negative result is still useful
If existing formats already perform well, or different formats suit different context types, that is more valuable than forcing the evidence to justify a new standard.
Related links.
Repositories, demos, event pages, posts, and other places connected to this project.
- Agent-Native Format repository and READMEPrimary source for the idea, personal motivation, current implementation, possible outcomes, and project boundaries.
- Benchmark v0 overviewDefines the task families, five baseline representations, required outputs, and measurement-first purpose.
- Benchmark v0 protocolDefines semantic equivalence, trial randomisation, repeated-read testing, evidence preservation, and fair analysis.
- Research question and scopeThe core research question, comparison boundaries, and success criteria.
- Benchmark metricsDefines accuracy, latency, resource, repeated-read, retrieval, and cost-per-success measurements.
- Threats to validityRecords risks including semantic drift, prompt effects, provider drift, caching, overfitting, and tokenizer mismatch.
- Design criteriaCandidate requirements for a useful agent-context format or container.