Regression Elo Arena¶
Across 18 benchmark datasets evaluated on test RMSE (round-robin pairwise, K=32).
Click any column header to sort. Sticky headers and frozen key columns on the left.
← Overall Arena | Classification Arena →
Executive Commentary & Analysis¶
1. Benchmark Methodology¶
- Protocol: 18 TabArena-v0.1 regression datasets spanning physical sciences (
superconductivity,airfoil,concrete), real-estate appraisal (houses,miami), biochemical assay (qsar_fish,qsar_tid11,protein), and commercial operations (food_delivery,fiat500). - Elo Rating Engine: Pairwise round-robin head-to-head matches on test RMSE (\(K=32\), initialized at \(R_0 = 1500\)). A lower test RMSE scores a win (\(S=1\)); identical RMSE scores a draw (\(S=0.5\)).
- Isolation Budget: Every model runs under strict serial single-job execution with a 300-second wall-clock cap per dataset.
2. Tier Analysis & Core Findings¶
- Tier 1 — Ensembles & Local Linear Leaves (Elo 1600–1725):
ChimeraBoost(1723.5),ranger(1685.4),sklearn RF(1670.4),ExtraTrees(1623.4), andgrf_llf(1585.4) lead the leaderboard.
Why local linear leaves excel: Standard axis-aligned trees approximate smooth continuous gradients with dozens of flat staircase steps, accumulating variance. In contrast, local linear models fit continuous slopes directly inside partitions, achieving low bias with far fewer splits. - Tier 2 — Gradient Boosted Decision Trees (Elo 1360–1525):
LightGBM(1525.9),perpetual(1490.2),CatBoost(1479.0), andforust(1363.6). Robust general-purpose performers, though requiring hundreds of sequential estimators. - Tier 3 — Interpretable Additive & Model Trees (Elo 850–1365):
EBM(1362.8),ModelTree(all)(1123.3),lmtree(1066.7),cubist(964.1), andModelTree(opt)(852.0). - The Speed-Accuracy Pareto Frontier:
WhileEBMachieves a strong Elo (1362.8), it requires 65.08 seconds per dataset on average. In sharp contrast,ModelTree(opt)executes in 0.10 seconds (~650x faster) while delivering transparent, auditable piecewise linear equations at every leaf.
| ELO | FIT TIME (±sd) | MODEL | abalone | boston | concrete | fc2008 | ozone | airfoil | ta_concrete | diamonds | fiat500 | food_delivery | healthcare | houses | miami | protein | qsar_fish | qsar_tid11 | superconductivity | wine |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1723.5 | 12.01±17.76s | ChimeraBoost 18/18 | 2.289 | 2.727 | 5.521 | 197.242 | 4.011 | 1.712 | 3.901 | 0.064 | 691.590 | 7.236 | 4425.975 | 0.208 | 85052.764 | 3.446 | 0.951 | 0.742 | 8.883 | 0.623 |
| 1685.4 | 18.24±23.58s | ranger 16/18 | 2.310 | 2.643 | 7.290 | 204.279 | 3.892 | · | 5.732 | 627.707 | 677.179 | 7.726 | 4634.886 | 0.238 | 101302.655 | 3.506 | · | 0.750 | 9.060 | 0.612 |
| 1670.4 | 19.20±33.35s | sklearn RF 18/18 | 2.298 | 3.216 | 6.060 | 201.606 | 3.881 | 1.838 | 4.847 | 0.171 | 684.020 | 7.929 | 4884.113 | 0.226 | 105171.270 | 3.538 | 0.942 | 0.743 | 9.153 | 0.613 |
| 1623.4 | 13.02±33.28s | ExtraTrees 18/18 | 2.345 | 2.804 | 6.062 | 202.052 | 4.099 | 1.430 | 4.600 | 0.066 | 693.526 | 8.640 | 5189.427 | 0.231 | 98024.414 | 3.392 | 0.930 | 1.027 | 8.771 | 0.602 |
| 1585.4 | 23.17±31.44s | grf_llf 16/18 | 2.238 | 3.760 | 6.314 | · | 3.804 | 2.344 | 5.128 | 565.086 | 698.664 | 7.550 | 4542.951 | 0.231 | 102561.418 | 3.593 | 0.966 | · | 8.641 | 0.638 |
| 1525.9 | 0.35±0.22s | LightGBM(200) 18/18 | 2.433 | 3.473 | 5.848 | 202.615 | 4.093 | 1.626 | 4.288 | 0.070 | 716.880 | 7.673 | 5148.259 | 0.216 | 93946.869 | 3.919 | 0.972 | 0.745 | 9.425 | 0.630 |
| 1490.2 | 7.20±23.69s | perpetual (Rust) 17/18 | 2.316 | 3.440 | 5.781 | 222.651 | 3.882 | 2.245 | 4.746 | 0.059 | 699.336 | 7.485 | 4537.289 | 0.211 | 91666.478 | 3.781 | 0.955 | · | 9.358 | 0.657 |
| 1479.0 | 1.04±1.10s | CatBoost(200) 18/18 | 2.308 | 2.503 | 5.739 | 196.293 | 3.817 | 1.707 | 4.055 | 0.069 | 691.069 | 7.434 | 4511.499 | 0.219 | 87832.800 | 3.923 | 0.951 | 0.764 | 9.918 | 0.648 |
| 1411.7 | 0.95±1.91s | HistGradientBoosting 17/18 | 2.362 | 3.318 | 5.878 | 200.246 | 3.836 | 1.778 | 4.559 | 0.069 | 714.725 | · | 4976.682 | 0.225 | 99388.451 | 4.106 | 0.966 | 0.774 | 10.200 | 0.650 |
| 1363.6 | 1.22±0.86s | forust (Rust) 18/18 | 2.424 | 3.152 | 5.587 | 167.737 | 3.941 | 1.606 | 4.428 | 0.089 | 747.923 | 7.435 | 4832.590 | 0.224 | 93775.195 | 3.965 | 1.012 | 0.780 | 9.680 | 0.658 |
| 1362.8 | 65.08±68.98s | EBM 15/18 | 2.300 | 3.420 | 5.287 | 163.233 | 3.724 | 1.850 | 3.748 | 0.060 | 689.761 | 7.447 | 4489.402 | 0.218 | 91918.766 | · | 0.973 | · | · | 0.663 |
| 1283.2 | 0.76±1.83s | CTBoost 17/18 | 2.502 | 4.014 | 6.376 | 193.047 | 4.543 | 2.529 | 4.907 | 0.065 | 707.656 | · | 4509.975 | 0.226 | 107014.090 | 4.193 | 0.990 | 0.834 | 10.987 | 0.686 |
| 1242.5 | 18.06±23.80s | grf_rf 18/18 | 2.359 | 3.265 | 8.261 | 232.406 | 4.482 | 2.792 | 6.500 | 594.072 | 700.546 | 7.547 | 4608.092 | 0.246 | 124681.686 | 3.997 | 0.979 | 0.932 | 10.772 | 0.693 |
| 1237.2 | 27.91±39.58s | bart 17/18 | 2.341 | 3.384 | 5.507 | 195.823 | 3.778 | 1.874 | 3.971 | 539.746 | 701.686 | 7.801 | 4487.422 | 0.234 | 101243.023 | 4.328 | 0.969 | · | 11.834 | 0.690 |
| 1123.3 | 13.77±20.73s | ModelTree(all) 16/18 | 2.409 | 4.537 | 6.421 | 265.250 | 4.311 | 3.290 | 6.485 | 0.062 | 789.071 | 7.420 | 4972.751 | 0.257 | 127066.860 | 4.516 | 0.998 | · | · | 0.705 |
| 1120.6 | 7.00±13.24s | sklearn GBM 18/18 | 2.291 | 3.209 | 6.377 | 190.926 | 3.881 | 2.622 | 5.380 | 0.068 | 700.354 | 7.640 | 4512.987 | 0.257 | 110163.529 | 4.702 | 0.962 | 0.961 | 12.615 | 0.694 |
| 1066.7 | 20.22±47.48s | lmtree 14/18 | 2.272 | 4.052 | 7.644 | · | 4.268 | 2.580 | 6.635 | 728.364 | 718.839 | · | 4486.791 | 0.238 | 113425.118 | · | 1.036 | 0.960 | · | 0.709 |
| 1065.6 | 15.29±51.54s | modeltrees lmtree 14/18 | 2.272 | 4.052 | 7.644 | · | 4.268 | 2.906 | 6.635 | · | 718.839 | · | 4486.791 | 0.238 | 104905.863 | 4.849 | 1.036 | 0.960 | · | 0.709 |
| 1033.1 | 29.02±57.11s | glmtree 14/18 | 2.276 | 4.050 | 7.425 | · | 4.268 | 2.557 | 6.131 | 626.466 | 718.839 | 8.013 | 4486.791 | 0.235 | · | · | 1.029 | 0.960 | · | 0.713 |
| 980.2 | 5.95±8.90s | ModelTree(winnow) 16/18 | 2.409 | 4.537 | 6.421 | 266.405 | 4.242 | 3.550 | 6.485 | 0.062 | 769.640 | 7.864 | 4972.751 | 0.329 | 126937.727 | 4.960 | 0.998 | · | · | 0.719 |
| 964.1 | 6.23±15.19s | cubist 17/18 | 2.294 | 3.980 | 6.812 | · | 4.572 | 2.473 | 6.144 | 638.234 | 755.036 | 8.418 | 4531.016 | 0.230 | 114607.603 | 4.432 | 1.020 | 0.881 | 12.451 | 0.723 |
| 885.3 | 20.49±31.78s | MARS (mars-earth) 10/18 | 2.274 | 4.845 | 7.113 | · | 4.100 | 4.397 | 6.382 | · | 708.531 | · | 5809.859 | · | · | · | 0.988 | · | · | 0.727 |
| 859.2 | 20.60±51.49s | mboost 15/18 | 2.364 | 3.522 | 7.144 | 234.631 | 3.786 | · | 6.153 | 816.077 | 722.724 | · | 4372.440 | 0.298 | 134404.244 | 5.035 | · | 1.081 | 15.321 | 0.720 |
| 852.0 | 0.10±0.11s | ModelTree(opt) 18/18 | 2.453 | 3.462 | 6.827 | inf | 4.535 | 2.949 | 6.467 | 0.091 | 775.779 | 7.590 | 4866.210 | 0.290 | 1353159.742 | 4.683 | 0.981 | 1.149 | 13.461 | 0.726 |
| 814.9 | 0.11±0.35s | ModelTree(v0.3.3-default) 18/18 | 2.434 | 3.571 | 6.929 | 871.944 | 3.949 | 3.969 | 6.262 | 0.014 | 805.629 | 9.969 | 4678.589 | 0.306 | 145375.963 | 4.836 | 1.049 | 0.966 | 20.572 | 0.713 |
| 767.4 | 3.19±7.11s | mltreelib MARS(deg=2) 17/18 | 2.346 | 3.257 | 7.063 | · | 4.128 | 3.586 | 6.231 | 0.060 | 1015.736 | 8.105 | 11046.034 | 0.295 | 137614.870 | 5.001 | 0.940 | 1.534 | 16.509 | 0.726 |
| 701.7 | 0.33±0.68s | statsmodels OLS 8/18 | · | · | 11.309 | · | 4.499 | · | 10.454 | · | · | · | · | 0.343 | · | 5.188 | 1.061 | 0.960 | 17.689 | · |
| 686.9 | 0.30±0.47s | ModelTree(leaf=mars,deg=1) 17/18 | · | 3.934 | 7.406 | 258.226 | 5.539 | 4.218 | 7.795 | 0.059 | 769.919 | 7.860 | 4586.386 | 0.317 | 147326.553 | 4.965 | 1.040 | 1.329 | 15.795 | 0.729 |
| 679.5 | 0.09±0.10s | KNN 18/18 | 2.504 | 5.759 | 10.947 | 250.524 | 5.496 | 6.149 | 9.070 | 0.189 | 829.638 | 9.519 | 11145.185 | 0.519 | 147140.199 | 5.888 | 1.037 | 0.864 | 11.804 | 0.804 |
| 677.4 | 0.74±1.38s | ModelTree(leaf=mars,deg=2) 17/18 | · | 4.232 | 7.259 | 258.226 | 5.809 | 3.663 | 12.073 | 0.060 | 1877.881 | 7.879 | 4786.108 | 0.323 | 144677.447 | 4.917 | 1.305 | 1.329 | 15.785 | 0.728 |
| 668.6 | 21.17±59.09s | ctree 17/18 | 2.417 | 4.019 | 9.757 | · | 4.260 | 3.788 | 7.098 | 603.732 | 780.636 | 7.875 | 4886.127 | 0.304 | 140802.325 | 4.594 | 1.078 | 1.059 | 12.731 | 0.746 |
| 656.8 | 0.75±1.30s | mltreelib MARS(deg=1) 17/18 | 2.294 | 4.374 | 7.242 | · | 4.864 | 4.706 | 6.603 | 0.060 | 718.256 | 8.119 | 10927.114 | 0.298 | 158430.436 | 5.065 | 1.042 | 1.534 | 17.667 | 0.728 |
| 654.5 | 10.19±27.50s | earth 17/18 | 2.259 | 4.454 | 7.060 | · | 4.228 | 4.311 | 6.197 | 1078.948 | 714.051 | 7.994 | 5724.629 | 0.291 | 145755.854 | 5.026 | 0.999 | 1.018 | 17.701 | 0.730 |
| 595.9 | 10.80±21.74s | ModelTree(path) 18/18 | 2.624 | 3.821 | 12.398 | 421.699 | 5.442 | 4.382 | 9.740 | 0.160 | 1130.844 | 7.598 | 4976.754 | 0.356 | 182240.202 | 5.332 | 1.140 | 1.331 | 15.398 | 0.740 |
| 583.4 | 14.37±7.33s | trust-free 4/18 | · | · | 7.511 | · | 4.471 | · | 6.646 | · | · | · | · | · | · | · | 1.087 | · | · | · |
| 562.4 | 0.34±0.57s | sklearn tree 18/18 | 3.237 | 4.769 | 7.931 | 210.311 | 5.456 | 2.801 | 5.665 | 0.110 | 854.893 | 10.593 | 6286.585 | 0.316 | 147951.144 | 5.068 | 1.273 | 1.057 | 11.611 | 0.846 |
| 554.9 | 0.09±0.23s | sklearn Linear 18/18 | 2.302 | 4.765 | 11.309 | 290.711 | 4.499 | 4.847 | 10.454 | 0.062 | 715.324 | 8.450 | 5854.983 | 0.343 | 189246.800 | 5.209 | 1.061 | 0.960 | 17.722 | 0.747 |
| 549.6 | 17.48±43.02s | statsmodels RLM 8/18 | · | · | 11.438 | · | 4.600 | · | 10.518 | · | · | · | · | 0.345 | · | 5.202 | 1.069 | 0.970 | 17.745 | · |
| 506.5 | 5.34±20.14s | ModelTree(mean) 18/18 | 2.519 | 4.341 | 11.056 | 292.703 | 5.485 | 4.818 | 9.893 | 0.090 | 780.544 | 7.838 | 4791.068 | 0.358 | 177412.312 | 5.220 | 1.112 | 1.274 | 16.798 | 0.747 |
| 370.2 | 22.52±59.15s | statsmodels QuantReg 7/18 | · | · | 12.037 | · | 4.979 | · | 10.867 | · | · | · | · | 0.345 | · | 5.374 | 1.079 | · | 17.985 | · |
| 335.1 | 1.34±3.71s | rpart 17/18 | 2.588 | 4.310 | 9.069 | · | 4.707 | 4.443 | 9.787 | 1296.423 | 823.833 | 8.323 | 5088.406 | 0.378 | 192132.402 | 5.459 | 1.124 | 1.267 | 18.245 | 0.774 |