Skip to content

Regression Elo Arena

Across 18 benchmark datasets evaluated on test RMSE (round-robin pairwise, K=32).
Click any column header to sort. Sticky headers and frozen key columns on the left.

← Overall Arena | Classification Arena →


Executive Commentary & Analysis

1. Benchmark Methodology

  • Protocol: 18 TabArena-v0.1 regression datasets spanning physical sciences (superconductivity, airfoil, concrete), real-estate appraisal (houses, miami), biochemical assay (qsar_fish, qsar_tid11, protein), and commercial operations (food_delivery, fiat500).
  • Elo Rating Engine: Pairwise round-robin head-to-head matches on test RMSE (\(K=32\), initialized at \(R_0 = 1500\)). A lower test RMSE scores a win (\(S=1\)); identical RMSE scores a draw (\(S=0.5\)).
  • Isolation Budget: Every model runs under strict serial single-job execution with a 300-second wall-clock cap per dataset.

2. Tier Analysis & Core Findings

  • Tier 1 — Ensembles & Local Linear Leaves (Elo 1600–1725):
    ChimeraBoost (1723.5), ranger (1685.4), sklearn RF (1670.4), ExtraTrees (1623.4), and grf_llf (1585.4) lead the leaderboard.
    Why local linear leaves excel: Standard axis-aligned trees approximate smooth continuous gradients with dozens of flat staircase steps, accumulating variance. In contrast, local linear models fit continuous slopes directly inside partitions, achieving low bias with far fewer splits.
  • Tier 2 — Gradient Boosted Decision Trees (Elo 1360–1525):
    LightGBM (1525.9), perpetual (1490.2), CatBoost (1479.0), and forust (1363.6). Robust general-purpose performers, though requiring hundreds of sequential estimators.
  • Tier 3 — Interpretable Additive & Model Trees (Elo 850–1365):
    EBM (1362.8), ModelTree(all) (1123.3), lmtree (1066.7), cubist (964.1), and ModelTree(opt) (852.0).
  • The Speed-Accuracy Pareto Frontier:
    While EBM achieves a strong Elo (1362.8), it requires 65.08 seconds per dataset on average. In sharp contrast, ModelTree(opt) executes in 0.10 seconds (~650x faster) while delivering transparent, auditable piecewise linear equations at every leaf.

ELO FIT TIME (±sd) MODELabalonebostonconcretefc2008ozoneairfoilta_concretediamondsfiat500food_deliveryhealthcarehousesmiamiproteinqsar_fishqsar_tid11superconductivitywine
1723.512.01±17.76sChimeraBoost 18/182.2892.7275.521197.2424.0111.7123.9010.064691.5907.2364425.9750.20885052.7643.4460.9510.7428.8830.623
1685.418.24±23.58sranger 16/182.3102.6437.290204.2793.892·5.732627.707677.1797.7264634.8860.238101302.6553.506·0.7509.0600.612
1670.419.20±33.35ssklearn RF 18/182.2983.2166.060201.6063.8811.8384.8470.171684.0207.9294884.1130.226105171.2703.5380.9420.7439.1530.613
1623.413.02±33.28sExtraTrees 18/182.3452.8046.062202.0524.0991.4304.6000.066693.5268.6405189.4270.23198024.4143.3920.9301.0278.7710.602
1585.423.17±31.44sgrf_llf 16/182.2383.7606.314·3.8042.3445.128565.086698.6647.5504542.9510.231102561.4183.5930.966·8.6410.638
1525.90.35±0.22sLightGBM(200) 18/182.4333.4735.848202.6154.0931.6264.2880.070716.8807.6735148.2590.21693946.8693.9190.9720.7459.4250.630
1490.27.20±23.69sperpetual (Rust) 17/182.3163.4405.781222.6513.8822.2454.7460.059699.3367.4854537.2890.21191666.4783.7810.955·9.3580.657
1479.01.04±1.10sCatBoost(200) 18/182.3082.5035.739196.2933.8171.7074.0550.069691.0697.4344511.4990.21987832.8003.9230.9510.7649.9180.648
1411.70.95±1.91sHistGradientBoosting 17/182.3623.3185.878200.2463.8361.7784.5590.069714.725·4976.6820.22599388.4514.1060.9660.77410.2000.650
1363.61.22±0.86sforust (Rust) 18/182.4243.1525.587167.7373.9411.6064.4280.089747.9237.4354832.5900.22493775.1953.9651.0120.7809.6800.658
1362.865.08±68.98sEBM 15/182.3003.4205.287163.2333.7241.8503.7480.060689.7617.4474489.4020.21891918.766·0.973··0.663
1283.20.76±1.83sCTBoost 17/182.5024.0146.376193.0474.5432.5294.9070.065707.656·4509.9750.226107014.0904.1930.9900.83410.9870.686
1242.518.06±23.80sgrf_rf 18/182.3593.2658.261232.4064.4822.7926.500594.072700.5467.5474608.0920.246124681.6863.9970.9790.93210.7720.693
1237.227.91±39.58sbart 17/182.3413.3845.507195.8233.7781.8743.971539.746701.6867.8014487.4220.234101243.0234.3280.969·11.8340.690
1123.313.77±20.73sModelTree(all) 16/182.4094.5376.421265.2504.3113.2906.4850.062789.0717.4204972.7510.257127066.8604.5160.998··0.705
1120.67.00±13.24ssklearn GBM 18/182.2913.2096.377190.9263.8812.6225.3800.068700.3547.6404512.9870.257110163.5294.7020.9620.96112.6150.694
1066.720.22±47.48slmtree 14/182.2724.0527.644·4.2682.5806.635728.364718.839·4486.7910.238113425.118·1.0360.960·0.709
1065.615.29±51.54smodeltrees lmtree 14/182.2724.0527.644·4.2682.9066.635·718.839·4486.7910.238104905.8634.8491.0360.960·0.709
1033.129.02±57.11sglmtree 14/182.2764.0507.425·4.2682.5576.131626.466718.8398.0134486.7910.235··1.0290.960·0.713
980.25.95±8.90sModelTree(winnow) 16/182.4094.5376.421266.4054.2423.5506.4850.062769.6407.8644972.7510.329126937.7274.9600.998··0.719
964.16.23±15.19scubist 17/182.2943.9806.812·4.5722.4736.144638.234755.0368.4184531.0160.230114607.6034.4321.0200.88112.4510.723
885.320.49±31.78sMARS (mars-earth) 10/182.2744.8457.113·4.1004.3976.382·708.531·5809.859···0.988··0.727
859.220.60±51.49smboost 15/182.3643.5227.144234.6313.786·6.153816.077722.724·4372.4400.298134404.2445.035·1.08115.3210.720
852.00.10±0.11sModelTree(opt) 18/182.4533.4626.827inf4.5352.9496.4670.091775.7797.5904866.2100.2901353159.7424.6830.9811.14913.4610.726
814.90.11±0.35sModelTree(v0.3.3-default) 18/182.4343.5716.929871.9443.9493.9696.2620.014805.6299.9694678.5890.306145375.9634.8361.0490.96620.5720.713
767.43.19±7.11smltreelib MARS(deg=2) 17/182.3463.2577.063·4.1283.5866.2310.0601015.7368.10511046.0340.295137614.8705.0010.9401.53416.5090.726
701.70.33±0.68sstatsmodels OLS 8/18··11.309·4.499·10.454····0.343·5.1881.0610.96017.689·
686.90.30±0.47sModelTree(leaf=mars,deg=1) 17/18·3.9347.406258.2265.5394.2187.7950.059769.9197.8604586.3860.317147326.5534.9651.0401.32915.7950.729
679.50.09±0.10sKNN 18/182.5045.75910.947250.5245.4966.1499.0700.189829.6389.51911145.1850.519147140.1995.8881.0370.86411.8040.804
677.40.74±1.38sModelTree(leaf=mars,deg=2) 17/18·4.2327.259258.2265.8093.66312.0730.0601877.8817.8794786.1080.323144677.4474.9171.3051.32915.7850.728
668.621.17±59.09sctree 17/182.4174.0199.757·4.2603.7887.098603.732780.6367.8754886.1270.304140802.3254.5941.0781.05912.7310.746
656.80.75±1.30smltreelib MARS(deg=1) 17/182.2944.3747.242·4.8644.7066.6030.060718.2568.11910927.1140.298158430.4365.0651.0421.53417.6670.728
654.510.19±27.50searth 17/182.2594.4547.060·4.2284.3116.1971078.948714.0517.9945724.6290.291145755.8545.0260.9991.01817.7010.730
595.910.80±21.74sModelTree(path) 18/182.6243.82112.398421.6995.4424.3829.7400.1601130.8447.5984976.7540.356182240.2025.3321.1401.33115.3980.740
583.414.37±7.33strust-free 4/18··7.511·4.471·6.646·······1.087···
562.40.34±0.57ssklearn tree 18/183.2374.7697.931210.3115.4562.8015.6650.110854.89310.5936286.5850.316147951.1445.0681.2731.05711.6110.846
554.90.09±0.23ssklearn Linear 18/182.3024.76511.309290.7114.4994.84710.4540.062715.3248.4505854.9830.343189246.8005.2091.0610.96017.7220.747
549.617.48±43.02sstatsmodels RLM 8/18··11.438·4.600·10.518····0.345·5.2021.0690.97017.745·
506.55.34±20.14sModelTree(mean) 18/182.5194.34111.056292.7035.4854.8189.8930.090780.5447.8384791.0680.358177412.3125.2201.1121.27416.7980.747
370.222.52±59.15sstatsmodels QuantReg 7/18··12.037·4.979·10.867····0.345·5.3741.079·17.985·
335.11.34±3.71srpart 17/182.5884.3109.069·4.7074.4439.7871296.423823.8338.3235088.4060.378192132.4025.4591.1241.26718.2450.774