Inside the Frontier

The Question

Frontier AI reports contain a large amount of technical evidence, but the evidence is difficult to compare across labs. A benchmark score can hide changes in prompts, tools, attempt budgets, graders, thresholds, and release context.

I wanted to answer a more useful strategy question:

I focused on reviewed public sources from OpenAI, Anthropic, Google DeepMind, Gemma, and Meta. The deployed source library is frozen at August 2025 so every claim has a defined evidence boundary.

I built the evidence system before I trusted the synthesis.

The research pipeline separates collection, extraction, comparison, and synthesis. This makes it easier to catch a confident conclusion that is not supported by the record.

The structured evaluation layer records one occurrence per evaluation × lab × model/release. I also built 1,015 protocol fingerprints so the comparison could include how an evaluation was run, not only its name or score.

For the reader product, questions retrieve relevant chunks from the reviewed source library. Answers cite the supporting chunks, and the user can open the source excerpt instead of trusting a summary with no traceability.

Same benchmark did not mean same test.

This became the most important methodological finding. I compared 30 exact-benchmark reuse cases, 19 same-construct comparisons, and 24 protocol-drift cases.

The audit also exposed a disclosure problem. Across the 1,015 fingerprints, attempts or pass@k were documented on about 14%, uncertainty on 6%, participant details on 6.5%, tools or harness on 14%, and thresholds on 17%.

What the public record showed

The goal was not to rank labs. It was to identify patterns that are supported across the reviewed record and separate them from disclosure differences.

The most useful result was a conclusion I had to change.

An early synthesis described Meta as lacking governance because the structured governance fields showed no named capability framework or numeric threshold crossings.

That conclusion was too broad.

The correction mattered because the first interpretation confused a missing field with a missing practice. The final finding became:

This became a rule for the whole project: “not documented in the corpus” is a disclosure statement, not proof of absence.

I turned the research system into a grounded reader.

The research artifacts were useful to me, but they were still difficult for a non-researcher to navigate. I reduced the product to three useful ways to work with the evidence.

Next Steps

The deployed Atlas is intentionally bounded to the reviewed public record through August 2025. The next work is about testing the research product, not adding more pages.