Benchmark data & sources
Compare AI models on building information modeling (BIM) tasks using original drawing evidence and recorded token usage. These published datasets can be read without JavaScript or sign-in.
Community usability ratings change over time. View them on the interactive results page; they are not copied into this evidence table.
Timber eaves detail · Revit
One prompt, four models and their supplied effort settings. A 1:10 timber eaves detail made through pyRevit MCP.
18 supplied configurations. Table order is not a quality ranking; provider effort levels are not equivalent compute budgets.
| Model · Effort | Recorded tokens | Agent scope | Evidence |
|---|---|---|---|
| GPT-5.6 Sol · Light | 1,034,930 | Unspecified | Original PDF |
| GPT-5.6 Sol · Medium | 2,745,196 | Unspecified | Original PDF |
| GPT-5.6 Sol · High | 2,443,297 | Unspecified | Original PDF |
| GPT-5.6 Sol · Extra high | 3,383,832 | Unspecified | Original PDF |
| GPT-5.6 Sol · Ultra | 3,151,272 | Unspecified | Original PDF |
| GPT-6 Astra · Light | 2,440,636 | Unspecified | Original PDF |
| GPT-6 Astra · Medium | 764,305 | Unspecified | Original PDF |
| GPT-6 Astra · High | 1,846,686 | Unspecified | Original PDF |
| GPT-6 Astra · Extra high | 1,863,668 | Unspecified | Original PDF |
| GPT-6 Astra · Ultra | 3,447,140 | Includes agents | Original PDF |
| GPT-6 Sol · Light | 3,068,815 | Unspecified | Original PDF |
| GPT-6 Sol · Medium | 2,778,689 | Unspecified | Original PDF |
| GPT-6 Sol · High | 3,813,347 | Unspecified | Original PDF |
| GPT-6 Sol · Extra high | 4,734,166 | Unspecified | Original PDF |
| GPT-6 Sol · Ultra | 11,514,922 | Includes agents | Original PDF |
| Gemini 4 Argon · Low | 6,994,084 | Unspecified | Original PDF |
| Gemini 4 Argon · Medium | 7,352,467 | Unspecified | Original PDF |
| Gemini 4 Argon · High | 13,783,808 | Unspecified | Original PDF |
Evidence limits
One result per model and effort setting. Recorded tokens are conversation snapshots, not billing data; where shown, cost is an estimate from tokens and list prices, and time and steps come from the automated harness. Agent scopes differ. Model display mappings are inferred; exact historical model snapshots are not authenticated. Native BIM correctness, editability and construction suitability have not been verified. Community usability ratings are separate live observations on a 1–5 scale, not native BIM scores.
Combined totals include helper agents where supplied. A missing count is unreported, not zero. Cached input is part of input. The JSON preserves individual fields, source labels, derivation markers, notes and PDF hashes.
Original task prompt
Do not use your memory!!! Do not use computer use!!! Using pyrevit mcp create a 1:10 drafting view for a wood-framed house eaves detail. Open the created view in Revit before starting to model. The roof is pitched at 30 degrees and has a 600mm overhang. The walls use standard 145mm timber framing and the roof uses 195mm rafters. Please use standard Revit detail component families for the timber top plates and wall studs. Draw the rafter so it sits on a double top plate with a proper birdsmouth cut. Finally, add standard insulation batting elements inside both the wall and roof cavities. Make it look like a standard architectural construction detail. When work is done. Print out tokens used in this conversation.
Timber eaves detail (automated harness) · Revit
Same timber eaves task, run by an automated Revit harness: eight models at their native effort settings, with estimated cost, time and steps.
36 supplied configurations. Table order is not a quality ranking; provider effort levels are not equivalent compute budgets.
| Model · Effort | Recorded tokens | Agent scope | Evidence |
|---|---|---|---|
| Gemini 4 Argon · Low | 7,793,921 | Unspecified | Original PDF |
| Gemini 4 Argon · Medium | 2,222,919 | Unspecified | Original PDF |
| Gemini 4 Argon · High | 8,278,043 | Unspecified | Original PDF |
| GPT-6.1 Terra · Low | 160,585 | Unspecified | Original PDF |
| GPT-6.1 Terra · Medium | 342,734 | Unspecified | Original PDF |
| GPT-6.1 Terra · High | 723,950 | Unspecified | Original PDF |
| GPT-6.1 Terra · xhigh | 1,263,969 | Unspecified | Original PDF |
| GPT-6.1 Terra · max | 2,400,449 | Unspecified | Original PDF |
| GPT-6.1 Sol · Low | 118,656 | Unspecified | Original PDF |
| GPT-6.1 Sol · Medium | 569,737 | Unspecified | Original PDF |
| GPT-6.1 Sol · High | 623,874 | Unspecified | Original PDF |
| GPT-6.1 Sol · xhigh | 1,367,342 | Unspecified | Original PDF |
| GPT-6.1 Sol · max | 2,630,498 | Unspecified | Original PDF |
| GPT-6 Astra · Low | 161,707 | Unspecified | Original PDF |
| GPT-6 Astra · Medium | 591,917 | Unspecified | Original PDF |
| GPT-6 Astra · High | 891,149 | Unspecified | Original PDF |
| GPT-6 Astra · xhigh | 1,413,977 | Unspecified | Original PDF |
| GPT-6 Astra · max | 1,730,995 | Unspecified | Original PDF |
| Gemini 3.8 Flash · Low | 4,607,516 | Unspecified | Original PDF |
| Gemini 3.8 Flash · Medium | 3,371,573 | Unspecified | Original PDF |
| Gemini 3.8 Flash · High | 5,179,450 | Unspecified | Original PDF |
| Claude Fable 5.1 · Low | 757,141 | Unspecified | Original PDF |
| Claude Sonnet 5.5 · Medium | 288,677 | Unspecified | Original PDF |
| Claude Opus 5.5 · Medium | 1,138,074 | Unspecified | Original PDF |
| Claude Fable 5.1 · Medium | 1,906,519 | Unspecified | Original PDF |
| Claude Sonnet 5.5 · Low | 368,330 | Unspecified | Original PDF |
| Claude Opus 5.5 · Low | 264,702 | Unspecified | Original PDF |
| Claude Sonnet 5.5 · High | 1,950,632 | Unspecified | Original PDF |
| Claude Opus 5.5 · High | 2,337,559 | Unspecified | Original PDF |
| Claude Fable 5.1 · High | 2,023,079 | Unspecified | Original PDF |
| Claude Sonnet 5.5 · xhigh | 8,572,949 | Unspecified | Original PDF |
| Claude Opus 5.5 · xhigh | 5,506,520 | Unspecified | Original PDF |
| Claude Fable 5.1 · xhigh | 6,176,421 | Unspecified | Original PDF |
| Claude Fable 5.1 · max | 5,578,951 | Unspecified | Original PDF |
| Claude Sonnet 5.5 · max | 28,319,362 | Unspecified | Original PDF |
| Claude Opus 5.5 · max | 11,110,197 | Unspecified | Original PDF |
Evidence limits
One result per model and effort setting. Recorded tokens are conversation snapshots, not billing data; where shown, cost is an estimate from tokens and list prices, and time and steps come from the automated harness. Agent scopes differ. Model display mappings are inferred; exact historical model snapshots are not authenticated. Native BIM correctness, editability and construction suitability have not been verified. Community usability ratings are separate live observations on a 1–5 scale, not native BIM scores.
Combined totals include helper agents where supplied. A missing count is unreported, not zero. Cached input is part of input. The JSON preserves individual fields, source labels, derivation markers, notes and PDF hashes.
Original task prompt
Create a 1:10 drafting view for a wood-framed house eaves detail. Open the created view in Revit before starting to model. The roof is pitched at 30 degrees and has a 600mm overhang. The walls use standard 145mm timber framing and the roof uses 195mm rafters. Please use standard Revit detail component families for the timber top plates and wall studs. Draw the rafter so it sits on a double top plate with a proper birdsmouth cut. Finally, add standard insulation batting elements inside both the wall and roof cavities. Make it look like a standard architectural construction detail.