Winning Gold at the IPhO

---

On July 8th, 2026, our AI system took the theory exam of the 56th International Physics Olympiad (IPhO) in Bucaramanga, Colombia. Our engineers traveled to Colombia to allow it to take the exam live during the same five-hour window as every other contestant, from 9:10 AM to 2:10 PM. The system was enclosed in an isolated environment, with no internet or access to external models. 2 days later, the official IPhO graders returned with our score: 28.6 points out of 30, a gold-medal performance. The IPhO committee formally recognized our results in the following month, issuing an official diploma in our name.

Why the IPhO?

We believe that scientific research depends on two broad capabilities: research intuition and raw problem-solving ability.

Research intuition is the ability to decide what to try, such as determining which questions matter, which hypotheses are promising, or how to make progress when there is no well-defined path forward. In our previous work with Thyla 1.0, we found that an agentic system built around open-weight models could already perform at the level of frontier models on open-ended research.

This left us wanting to understand how far similar agentic systems built on open-weight models could perform on tasks that primarily test raw intelligence, compared to frontier models.

The IPhO was our method to test that, as its theory exam is substantially difficult and graded against a fixed standard. Unlike an open-ended research benchmark, the objective is to solve well-defined physics problems correctly, under time pressure with no external information. With these conditions, we wanted to see how far an open-weight model could go when placed inside another agentic system.

How the system works

Our system was built as a fully autonomous agentic framework, with its core reasoning loop built on the open-weight Qwen3.5-397B-A17B model. Three key properties shaped our design:

• End-to-end. The framework operates the entire pipeline, including input PDF scanning, reasoning, self-verification, peer review, and output PDF typesetting.
• Fully isolated execution. Reasoning branches run within their own disposable sandbox, allowing model-written code to be executed freely without ever touching the host or another branch.
• Isolated and offline. The entire system runs entirely on self-hosted open-weight models, in an isolated environment with no internet access.

The system ran on AWS cloud infrastructure, in an isolated environment with no internet access. Within this environment, our pipeline orchestrated multiple self-hosted models, each taking on different roles to contribute to a final solution.

Isolation ensured that neither the exam nor the intermediate reasoning ever left our environment, making it suitable to process sensitive or embargoed material without relying on a third party vendor. Self-hosted pinned models also made our runs reproducible, insulating the system from unexpected upstream changes.

Overview

The system ran in six main steps:

1. OCR: The exam PDF is translated into a JSONL file.
2. Preprocessing: Figures are captioned with a vision-capable LLM.
3. Branched solve: Multiple reasoning branches are generated across all questions.
4. Self-verification: The LLM audits its own solutions in a Python sandbox.
5. Peer review: A panel of 11 agents reviews solutions, each enforcing a single constraint.
6. Typesetting: The final solution PDF is produced through LaTeX.

Processing the input

Our system operated directly on the scanned exam PDF. Therefore, we first converted the exam PDF into a structured JSONL representation using open-source OCR models. Then, we used a vision-capable LLM to caption figures and attach descriptions that are more suitable for downstream text-only reasoning.

Not betting everything on a single pass

For each question, the solver then generated multiple branches of reasoning. A separate arbiter agent selected the strongest branch so far, pruning the remaining branches.

In the real IPhO run, three branches were generated per question, which led to the best balance between accuracy and speed. Internal experiments with one to five branches showed that fewer branches destabilized accuracy, whereas more branches overwhelmed the arbiter and excessively delayed answer generation.

A sandboxed environment for each branch

Each reasoning branch then received disposable Linux microVMs, where the agent could verify its work by writing and executing Python code. This ensured that one branch’s state could never contaminate another’s, and freely allowed the execution of unstrusted code without risk to the host.

Adversarial peer review

We then ran a panel of critic sub-agents, each responsible for one axis among eleven evaluation criteria:

1. Math formatting: Every displayed equation and final answer is wrapped in proper math delimiters.
2. Requested figures: The solution includes a sketch or plot if requested.
3. Completeness: No sub-part is skipped or left partially answered.
4. Case notation: Conditions in piecewise expressions are correctly parenthesized.
5. Variable discipline: All variables are defined, and no variable misuse arises from unrelated sub-questions.
6. Numerical evaluation: A final answer is reduced to a single number if possible.
7. Dimensional consistency: The units on both sides of every derived equation match.
8. Sign convention: Change is computed as the final value minus the initial value instead of the reverse.
9. Algebraic integrity: No constant or factor is dropped or introduced across a derivation.
10. Significant figures: Final numerical answers carry a precision consistent with the least-precise given datum.
11. Derivation integrity: The final expression follows from the solution’s own complete derivation, and does not appeal to a “standard” or “known” result. Derivation integrity was necessary because IPhO grading may penalize the use of memorized formulas that are not considered standard results.

Here, every sub-agent must approve before a result is accepted. If any critic is uncertain, the reasoning branch failed, and the pipeline continuously repeated generation until peer review passed.

The system’s output

Our system scored 28.6 points out of 30, a loss of 1.4 points. Below is the system’s actual full output, along with the specific locations where the official graders deducted points.

The 1.4-point deduction

Decomposing the deductions, 1.3 points were lost in T2-A1, and 0.1 point was lost in T3-C3.

• In T2-A1, the system made extensive use of the given sandboxing tool and performed a numerical simulation. While the answer was correct, the IPhO grader stated that such a numerical simulation was not a feasible solution for human solvers. Therefore, only 0.2 points were awarded out of 1.5.
• In T3-C3, the system computed and stated the correct final value with the correct number of significant figures. But the model then added extra subtext, justifying why the answer had 2 significant figures. The model attributed the significant-figure count to the second term of the final arithmetic operation, when it was the first term that determined it. As a result, 0.7 points were awarded out of 0.8.

In both cases, our system found the correct answers, but was deducted points due to solution-method feasibility and incorrect justifications.

Notes

In T2-B2, our model developed an alternative solution that bypassed the intermediate checkpoints at which the official solution awards partial marks. The bypass resulted in a 1.3-point loss, as it failed to show the checkpoints defined in the official marking schema. However, the model’s solution was accepted as an alternative with no logical gaps during the moderation session, receiving full marks.

Our takeaway

To us, the IPhO result complements what we saw with Thyla. There, an agentic system built on open-weight models showed that it could make progress on open-ended research tasks, where choosing what to try is a critical part of the pipeline. By comparison, here we tested the same general approach using a more direct measure of technical problem-solving. A score of 28.6 out of 30 suggests that constructing agentic systems around weaker models can still perform at the frontier level, even when the task primarily tests raw reasoning ability.

What matters most to us is that this performance did not come solely from the base model. Better models will continue to matter, but we believe that there is still substantial capability to be gained from system design itself. Taken together with Thyla, this result strengthens our conviction that the frontier is not defined by model weights alone, and that open-weight models can already support systems that operate well beyond their standalone capabilities.