Overall Benchmark Arena¶
Cumulative Elo leaderboard combining all 56 benchmark datasets (18 Regression on test RMSE + 38 Classification on test Accuracy).
Click any column header to sort. Evaluates models across Rust, Python, and R under identical 300s wall-clock isolation.
Arena Navigation:
- Regression Arena (18 datasets)
- Classification Arena (38 datasets)
- Datasets Metadata & Sources
Executive Summary & Cross-Task Insights¶
1. Unified Arena Overview¶
The cumulative TabArena-v0.1 arena provides an empirical tournament across 56 diverse datasets spanning physical sciences, medical diagnosis, financial churn, industrial failures, and web analytics. Every algorithm was tested under strict serial evaluation with a 300-second wall-clock timeout per run.
2. High-Altitude Rankings¶
- grf_llf (Rank 1, Elo 1585.4): Leads in regression via Generalized Random Forests with local linear leaves.
- CatBoost (Rank 2, Overall Elo 1477.5): The most consistent dual-domain performer across all 56 datasets (Reg: 1479.0, Cla: 1388.0).
- perpetual (Rust) (Rank 3, Overall Elo 1391.8): High-speed pure-Rust gradient booster with automatic stopping and zero hyperparameters.
- sklearn RF & ChimeraBoost (Ranks 4-5, Elo 1368.5 & 1354.9): Random forests remain extraordinarily resilient across diverse tabular domains without manual tuning.
- EBM (Explainable Boosting Machine) (Rank 6, Elo 1353.5): Highest-ranking fully interpretable model, though computationally expensive (~44.7s average fit time).
- ModelTree(all) (Rank 7, Overall Elo 1333.9): The top-performing pure-Rust model tree architecture. It outperforms traditional trees (
ctree,rpart,sklearn tree) by 300+ Elo points and matches or exceeds standard gradient boosters on many tasks by substituting flat leaf constants with regularized local linear models.
3. Language & Runtime Economy¶
- Rust Core Advantage:
ModelTreeandperpetualprovide native memory safety and multi-threaded throughput with zero reliance on external BLAS/Fortran runtimes. - Microsecond Leaf Updates: While full GBDT re-fitting requires dozens of seconds,
ModelTreeenables frozen-topology leaf retraining in < 2 ms, allowing high-throughput online serving on streaming data.
Overall Leaderboard (56 Datasets)¶
| RANK | OVERALL ELO | MODEL | REG ELO | CLA ELO | DATASETS | FIT TIME (±sd) | LANG | CAT HANDLING |
|---|---|---|---|---|---|---|---|---|
| 1 | 1585.4 | grf_llf | 1585.4 | · | 16/56 | 23.17±31.44s | R | none |
| 2 | 1477.5 | CatBoost(200) | 1479.0 | 1388.0 | 56/56 | 5.59±19.65s | Python | internal |
| 3 | 1391.8 | perpetual (Rust) | 1490.2 | 1302.3 | 41/56 | 15.04±30.44s | Rust | external |
| 4 | 1368.5 | sklearn RF | 1670.4 | 1279.0 | 56/56 | 9.76±22.40s | Python | external |
| 5 | 1354.9 | ChimeraBoost | 1723.5 | 1265.3 | 56/56 | 7.42±11.86s | Python | internal |
| 6 | 1353.5 | EBM | 1362.8 | 1264.0 | 44/56 | 44.66±49.61s | Python | internal |
| 7 | 1333.9 | ModelTree(all) | 1123.3 | 1244.3 | 54/56 | 17.13±36.34s | Rust | internal |
| 8 | 1318.3 | bart | 1237.2 | 1228.8 | 36/56 | 19.07±29.65s | R | internal |
| 9 | 1293.5 | ranger | 1685.4 | 1204.0 | 54/56 | 18.10±30.97s | R | internal |
| 10 | 1281.7 | sklearn GBM | 1120.6 | 1192.2 | 55/56 | 8.77±19.04s | Python | external |
| 11 | 1277.4 | c50_rules | · | 1187.8 | 33/56 | 3.99±5.63s | R | internal |
| 12 | 1260.3 | HistGradientBoosting | 1411.7 | 1170.8 | 50/56 | 1.34±3.09s | Python | internal |
| 13 | 1248.9 | c50_tree | · | 1159.4 | 36/56 | 5.00±7.76s | R | internal |
| 14 | 1237.7 | LightGBM(200) | 1525.9 | 1148.2 | 56/56 | 0.60±0.80s | Python | internal |
| 15 | 1200.4 | CTBoost | 1283.2 | 1110.9 | 46/56 | 0.65±1.56s | Python | internal |
| 16 | 1165.8 | ExtraTrees | 1623.4 | 1076.2 | 56/56 | 5.77±19.44s | Python | external |
| 17 | 1126.4 | forust (Rust) | 1363.6 | 1036.9 | 48/56 | 1.19±0.81s | Rust | external |
| 18 | 1108.3 | MARS (mars-earth) | 885.3 | 1018.8 | 18/56 | 27.24±40.59s | Python | external |
| 19 | 1066.7 | lmtree | 1066.7 | · | 14/56 | 20.22±47.48s | R | internal |
| 20 | 1065.6 | modeltrees lmtree | 1065.6 | · | 14/56 | 15.29±51.54s | Python | internal |
| 21 | 1037.7 | ctree | 668.6 | 948.1 | 44/56 | 9.71±37.27s | R | internal |
| 22 | 1036.6 | ModelTree(opt) | 852.0 | 947.1 | 56/56 | 2.37±8.95s | Rust | internal |
| 23 | 1015.2 | ModelTree(path) | 595.9 | 925.6 | 56/56 | 8.23±31.06s | Rust | internal |
| 24 | 1013.9 | ModelTree(mean) | 506.5 | 924.4 | 56/56 | 6.11±30.50s | Rust | internal |
| 25 | 999.1 | grf_rf | 1242.5 | 909.5 | 27/56 | 15.45±20.17s | R | none |
| 26 | 964.1 | cubist | 964.1 | · | 17/56 | 6.23±15.19s | R | internal |
| 27 | 925.7 | ModelTree(winnow) | 980.2 | 836.1 | 54/56 | 6.54±10.19s | Rust | internal |
| 28 | 889.7 | sklearn tree | 562.4 | 800.1 | 56/56 | 0.85±3.85s | Python | external |
| 29 | 880.5 | KNN | 679.5 | 791.0 | 56/56 | 0.26±0.53s | Python | external |
| 30 | 849.9 | rpart | 335.1 | 760.4 | 53/56 | 2.02±5.98s | R | internal |
| 31 | 827.3 | modeltrees glmtree | · | 737.7 | 16/56 | 23.99±39.03s | Python | internal |
| 32 | 816.8 | statsmodels MNLogit | · | 727.6 | 5/56 | 0.03±0.01s | Python | none |
| 33 | 814.9 | ModelTree(v0.3.3-default) | 814.9 | · | 18/56 | 0.11±0.35s | Rust | internal |
| 34 | 790.9 | mboost | 859.2 | 701.3 | 34/56 | 16.11±37.99s | R | internal |
| 35 | 790.7 | modeltrees multinomtree | · | 701.1 | 15/56 | 11.53±25.66s | Python | internal |
| 36 | 777.4 | glmtree | 1033.1 | 687.9 | 30/56 | 28.38±48.39s | R | internal |
| 37 | 767.4 | mltreelib MARS(deg=2) | 767.4 | · | 17/56 | 3.19±7.11s | Python | unknown |
| 38 | 756.3 | earth | 654.5 | 666.8 | 36/56 | 5.92±19.25s | R | internal |
| 39 | 748.3 | sklearn LogReg | · | 658.7 | 38/56 | 4.27±6.32s | Python | external |
| 40 | 701.7 | statsmodels OLS | 701.7 | · | 8/56 | 0.33±0.68s | Python | none |
| 41 | 686.9 | ModelTree(leaf=mars,deg=1) | 686.9 | · | 17/56 | 0.30±0.47s | Rust | internal |
| 42 | 677.4 | ModelTree(leaf=mars,deg=2) | 677.4 | · | 17/56 | 0.74±1.38s | Rust | internal |
| 43 | 656.8 | mltreelib MARS(deg=1) | 656.8 | · | 17/56 | 0.75±1.30s | Python | unknown |
| 44 | 583.4 | trust-free | 583.4 | · | 4/56 | 14.37±7.33s | Python | none |
| 45 | 554.9 | sklearn Linear | 554.9 | · | 18/56 | 0.09±0.23s | Python | external |
| 46 | 549.6 | statsmodels RLM | 549.6 | · | 8/56 | 17.48±43.02s | Python | none |
| 47 | 370.2 | statsmodels QuantReg | 370.2 | · | 7/56 | 22.52±59.15s | Python | none |