Fidelity Isn't Accuracy: When Linearly Decodable Functions Fail to Match the Ground Truth

Benchmark Model Rank Results
regression-on-california-housing-pricesNeural Network#1R2 Score: 0.7908lambda: 0.6968
regression-on-california-housing-pricesBaseline Regression#2R2 Score: 0.5758
regression-on-california-housing-pricesMimic / Surrogate#3R2 Score: 0.5658
regression-on-concrete-compressive-strengthNeural Network#1R2 Score: 0.8588lambda: 0.6659
regression-on-concrete-compressive-strengthBaseline Regression#2R2 Score: 0.5944
regression-on-concrete-compressive-strengthMimic / Surrogate#3R2 Score: 0.5821
regression-on-medical-cost-personal-datasetNeural Network#1R2 Score: 0.8673lambda: 0.9186
regression-on-medical-cost-personal-datasetBaseline Regression#2R2 Score: 0.7836
regression-on-medical-cost-personal-datasetMimic / Surrogate#3R2 Score: 0.7835
regression-on-synthetic-y-x-sin-xNeural Network#1R2 Score: 0.9755lambda: -0.0105
regression-on-synthetic-y-x-sin-xBaseline Regression#2R2 Score: -0.0080
regression-on-synthetic-y-x-sin-xMimic / Surrogate#3R2 Score: -0.0137