What nine models asked for in repository metadata

October 8, 2026 correction note: The interpretation was revised in September: model agreement supplies design hypotheses, not independent empirical confirmation. Self-reported reasoning costs and adoption thresholds remain suggestions, not measurements. This note also narrows the session description to the retained account of logged-in and fresh sessions; it does not establish a controlled memory setting. The original March 18 publication date and response counts are retained.

dotrepo is a metadata protocol for software repositories. One of its three intended audiences is AI coding agents — models that get dropped into unfamiliar repos and have to figure out what the project is, how to build it, who owns it, and what constraints apply.

I wanted to know which repository questions models found difficult and what information they thought would help. I sent the same prompt to nine models across twelve sessions to collect ideas for dotrepo.

The setup

I wrote a 10-question interview prompt covering first impressions, schema design, trust, adoption, and failure modes. The last question was “If you could say one thing directly to the person building this — advice, encouragement, warning, request — what would it be?”

The models interviewed: ChatGPT 5.4 Thinking, Claude Opus 4.6 Extended, Gemini Pro 3.1, Gemini Thinking 3.1, Grok Expert 4.20, GLM-5, Hunter Alpha, MiniMax M2.5, and Nemotron 3 Super. The retained public method describes ChatGPT, Claude, and Grok each appearing in a logged-in session and a fresh session. It does not document a controlled memory configuration for every provider. Several models searched the repository on the web before answering.

I asked for critical feedback and used the same prompt verbatim. These were interviews, not tests of the models doing repository work. The counts below describe their suggestions; agreement between models is not independent confirmation.

Build commands, sources, and stale records

All nine models described figuring out how to build and test a project as the most expensive part of entering an unfamiliar repository. Their examples concerned the gap between finding a build file and choosing the correct command, including flags, prerequisites, environment variables, and side effects. Several described spending 3–5 reasoning turns working out the command from CI YAML. Those were estimates in their responses, not timings I measured.

All nine highlighted the overlay index, which provides metadata for repositories that have not adopted dotrepo. Their argument was practical: a new consumer needs useful records before many maintainers have installed the tool.

All nine also highlighted the distinction between maintainer-declared facts, imported values, and inferred guesses. They described using those categories to decide when to inspect source files or seek confirmation.

All nine warned about stale metadata. The concern was that an apparently authoritative .repo file could discourage the source checks an agent would otherwise make.

Suggested schema and tooling changes

All twelve sessions asked for structured build and test commands containing prerequisites, environment requirements, platform constraints, and a flag for agent execution safety. A safety flag needs particular care: neither it nor a provenance label can account for every environment or grant permission to run a command.

Ten of the twelve sessions asked for monorepo and workspace semantics. Seven asked for path semantics — explicit “generated / vendored / do-not-touch” annotations. Seven asked for structured contribution workflow metadata (CLA required? DCO? PR process?). Five pushed for field-level trust provenance instead of record-level.

On the MCP server side, six of the twelve sessions asked for remote lookup: take a repository URL and return structured metadata without requiring a local clone. Several ranked this as the most useful missing capability.

The responses suggested that roughly 2,000–5,000 high-quality records for commonly encountered open source projects might make routine lookup worthwhile. These numbers are speculative adoption estimates, not evidence of how often agents would use the index or how much work it would save.

What the models disagreed on

MiniMax gave the longest adoption estimate: 5–10 years even if things went well. Other models suggested 30-day or 90-day actions to start building use. Those answers concern different milestones, and none establishes a reliable adoption forecast.

Gemini Pro proposed an ai_hints schema field where maintainers could leave notes explicitly for AI (“do not touch the legacy parser in /old”). Most other models did not ask for this, preferring structured metadata over freeform directives.

Nemotron pushed hard for environment-aware metadata — build profiles, feature flags, platform-conditional commands — while most others wanted the schema to stay static and simple.

The trust provenance granularity question split roughly 5–7. Five of the twelve sessions pushed for field-level provenance (each fact tracked individually). Seven were comfortable with record-level provenance, at least for now. The five who wanted field-level made the stronger argument — a record can have declared identity but inferred build commands — but the seven had a pragmatic point about implementation overhead.

The priorities I took from the interviews

At the time of these March interviews, I took away three priorities. They describe that stage of the project; the project page covers what is available now.

First, expand the index beyond its five overlay records. I took 500+ high-quality records across several languages as an initial target, distinct from the larger adoption estimates in the responses.

Second, add remote lookup to MCP. The public HTTP endpoints already served metadata and trust information, but the MCP server did not yet expose that lookup.

Third, keep the initial scope focused on build commands, documentation, ownership, security contacts, and project status. The responses repeatedly warned that a wider schema would be harder to keep accurate.

What I want to test

I want to compare agents completing the same repository tasks with and without explicit build prerequisites, sources and check dates, and remote lookup. Do they choose the right commands, catch stale or conflicting answers, and spend less time and computation finding the information they need?

The interviews helped me choose what to build, but they don’t answer those questions. The sessions shared a prompt and three models appeared twice. The retained method does not control session context or web access. The models may also share training data or design choices. A task comparison would need to control for those differences and measure actual behavior.

The public synthesis retains the model list and response counts. The project page links the current toolchain and evidence. If you build coding tools or maintain a repository and want to try dotrepo, get in touch.