This experiment is planned.
No runs have been published yet.
Experiment results
↕ Sort 2
About these results +
Usability is the average of individual 1–5 ratings: 1 Unusable, 2 Major rework, 3 Usable with substantial edits, 4 Usable with minor edits, 5 Ready to use as a drawing. Cannot judge is excluded. This is perceived drawing usability, not BIM correctness or approval for construction. † Includes reported helper agents; other agent scopes are unspecified. Token counts are not billed cost.
Drawing order groups models and effort settings; it is not a quality ranking. Effort bars use each model’s own settings: five for the OpenAI runs and three for Argon. Low and Light retain their source labels; effort levels across providers are not equal compute budgets. Full-sheet previews use varying paper sizes, not a shared physical scale.
Exact token counts & sources+
| Model · Effort | Total | Input | Cached input¹ | Output | Agent scope |
|---|
¹ Cached input is part of input; reasoning is part of output. — = unreported. * = arithmetic derivation. GPT-6 Sol · Ultra uses the reported combined total and combined cached count; input/output are unreported. Its separate chat-only snapshot is 7,119,141.
Drawings
Vote before viewing the drawings
Rate an anonymous drawing first, or browse the PDFs now.
Vote first ↗Browsing first may influence your later votes.
The test prompt+
What these results can tell us+
One result per model and effort setting. The PDFs show the visible drawing; native element identity, editability and execution-rule compliance have not been verified. Token figures are snapshots, with differing agent scopes. Community ratings describe perceived drawing usability; no native BIM score or billing data is inferred. Where shown, cost is an estimate from tokens and list prices, and time and steps come from the automated harness.
Full method and future tests ↗Call for support
Help fund independent tests
We spent approx. $140 in API token costs running the first two eaves detail benchmarks (54 runs across frontier models), and the upcoming full 3D house benchmark is estimated at another $110, bringing compute to around $250 in total. Support covers API token costs and compute for running real Revit models without vendor bias. BIM Bench is completely independent, with no corporate sponsors or vendor affiliations. Every benchmark is open evidence and community-rated.