CHILI: Chemically-Informed Large-scale Inorganic Nanomaterials Dataset for Advancing Graph Machine Learning

Open paper
Benchmark Model Rank Results
atomic-number-classification-on-chili-100kEdgeCNN#1F1-score (Weighted): 0.572 +/- 0.017
atomic-number-classification-on-chili-100kGIN#2F1-score (Weighted): 0.336 +/- 0.005
atomic-number-classification-on-chili-100kGraphUNet#3F1-score (Weighted): 0.287 +/- 0.004
atomic-number-classification-on-chili-100kGCN#4F1-score (Weighted): 0.275 +/- 0.002
atomic-number-classification-on-chili-100kGraphSAGE#5F1-score (Weighted): 0.195 +/- 0.007
atomic-number-classification-on-chili-100kMost Frequent Class#6F1-score (Weighted): 0.192
atomic-number-classification-on-chili-100kGAT#7F1-score (Weighted): 0.192 +/- 0.000
atomic-number-classification-on-chili-100kPMLP#8F1-score (Weighted): 0.191 +/- 0.000
atomic-number-classification-on-chili-100kRandom#9F1-score (Weighted): 0.015 +/- 0.000
atomic-number-classification-on-chili-3kEdgeCNN#1F1-score (Weighted): 0.632 +/- 0.009
atomic-number-classification-on-chili-3kGIN#2F1-score (Weighted): 0.587 +/- 0.002
atomic-number-classification-on-chili-3kGraphUNet#3F1-score (Weighted): 0.552 +/- 0.079
atomic-number-classification-on-chili-3kGCN#4F1-score (Weighted): 0.496 +/- 0.001
atomic-number-classification-on-chili-3kGraphSAGE#5F1-score (Weighted): 0.491 +/- 0.004
atomic-number-classification-on-chili-3kMost Frequent Class#6F1-score (Weighted): 0.461
atomic-number-classification-on-chili-3kPMLP#7F1-score (Weighted): 0.461 +/- 0.000
atomic-number-classification-on-chili-3kGAT#8F1-score (Weighted): 0.461 +/- 0.000
atomic-number-classification-on-chili-3kRandom#9F1-score (Weighted): 0.016 +/- 0.000
crystal-system-classification-on-chili-100kRandom#1F1-score (Weighted): 0.168 +/- 0.014
crystal-system-classification-on-chili-100kPMLP#2F1-score (Weighted): 0.124 +/- 0.036
crystal-system-classification-on-chili-100kGAT#3F1-score (Weighted): 0.110 +/- 0.029
crystal-system-classification-on-chili-100kEdgeCNN#4F1-score (Weighted): 0.072 +/- 0.047
crystal-system-classification-on-chili-100kGCN#5F1-score (Weighted): 0.069 +/- 0.023
crystal-system-classification-on-chili-100kGIN#6F1-score (Weighted): 0.069 +/- 0.040
crystal-system-classification-on-chili-100kGraphUNet#7F1-score (Weighted): 0.068 +/- 0.006
crystal-system-classification-on-chili-100kGraphSAGE#8F1-score (Weighted): 0.061 +/- 0.019
crystal-system-classification-on-chili-100kMost Frequent Class#9F1-score (Weighted): 0.046
crystal-system-classification-on-chili-3kEdgeCNN#1F1-score (Weighted): 0.657 +/- 0.196
crystal-system-classification-on-chili-3kGAT#2F1-score (Weighted): 0.504 +/- 0.076
crystal-system-classification-on-chili-3kMost Frequent Class#3F1-score (Weighted): 0.440
crystal-system-classification-on-chili-3kPMLP#4F1-score (Weighted): 0.440 +/- 0.036
crystal-system-classification-on-chili-3kGIN#5F1-score (Weighted): 0.438 +/- 0.004
crystal-system-classification-on-chili-3kGraphUNet#6F1-score (Weighted): 0.431 +/- 0.014
crystal-system-classification-on-chili-3kGraphSAGE#7F1-score (Weighted): 0.422 +/- 0.037
crystal-system-classification-on-chili-3kGCN#8F1-score (Weighted): 0.367 +/- 0.127
crystal-system-classification-on-chili-3kRandom#9F1-score (Weighted): 0.191 +/- 0.008
distance-regression-on-chili-100kEdgeCNN#1MSE: 0.030 +/- 0.001
distance-regression-on-chili-100kGraphSAGE#2MSE: 0.064 +/- 0.001
distance-regression-on-chili-100kGraphUNet#3MSE: 0.085 +/- 0.002
distance-regression-on-chili-100kGCN#4MSE: 0.090 +/- 0.002
distance-regression-on-chili-100kGAT#5MSE: 0.252 +/- 0.003
distance-regression-on-chili-100kMean#6MSE: 0.307
distance-regression-on-chili-100kPMLP#7MSE: 0.486 +/- 0.014
distance-regression-on-chili-100kGIN#8MSE: 0.491 +/- 0.038
distance-regression-on-chili-3kEdgeCNN#1MSE: 0.015 +/- 0.001
distance-regression-on-chili-3kGraphSAGE#2MSE: 0.055 +/- 0.002
distance-regression-on-chili-3kGraphUNet#3MSE: 0.055 +/- 0.001
distance-regression-on-chili-3kGCN#4MSE: 0.056 +/- 0.006
distance-regression-on-chili-3kMean#5MSE: 0.265
distance-regression-on-chili-3kGAT#6MSE: 0.342 +/- 0.117
distance-regression-on-chili-3kPMLP#7MSE: 0.359 +/- 0.017
distance-regression-on-chili-3kGIN#8MSE: 0.464 +/- 0.005
position-regression-on-chili-100kGraphUNet#1Positional MAE: 14.824 +/- 0.315
position-regression-on-chili-100kMean#2Positional MAE: 16.336
position-regression-on-chili-100kGCN#3Positional MAE: 16.336 +/- 0.000
position-regression-on-chili-100kPMLP#4Positional MAE: 16.336 +/- 0.000
position-regression-on-chili-100kGAT#5Positional MAE: 16.336 +/- 0.000
position-regression-on-chili-100kGIN#6Positional MAE: 16.336 +/- 0.000
position-regression-on-chili-100kEdgeCNN#7Positional MAE: 16.336 +/- 0.000
position-regression-on-chili-100kGraphSAGE#8Positional MAE: 16.337 +/- 0.000
position-regression-on-chili-3kGraphUNet#1Positional MAE: 14.765 +/- 0.395
position-regression-on-chili-3kMean#2Positional MAE: 16.575
position-regression-on-chili-3kGCN#3Positional MAE: 16.575 +/- 0.000
position-regression-on-chili-3kPMLP#4Positional MAE: 16.575 +/- 0.000
position-regression-on-chili-3kGraphSAGE#5Positional MAE: 16.575 +/- 0.000
position-regression-on-chili-3kGAT#6Positional MAE: 16.575 +/- 0.000
position-regression-on-chili-3kGIN#7Positional MAE: 16.575 +/- 0.000
position-regression-on-chili-3kEdgeCNN#8Positional MAE: 16.575 +/- 0.000
saxs-regression-on-chili-100kPMLP#1MSE: 0.003 +/- 0.000
saxs-regression-on-chili-100kEdgeCNN#2MSE: 0.007 +/- 0.009
saxs-regression-on-chili-100kGAT#3MSE: 0.009 +/- 0.000
saxs-regression-on-chili-100kGraphUNet#4MSE: 0.009 +/- 0.000
saxs-regression-on-chili-100kGIN#5MSE: 0.009 +/- 0.000
saxs-regression-on-chili-100kGCN#6MSE: 0.010 +/- 0.000
saxs-regression-on-chili-100kGraphSAGE#7MSE: 0.011 +/- 0.002
saxs-regression-on-chili-100kMean#8MSE: 0.038
saxs-regression-on-chili-3kEdgeCNN#1MSE: 0.006 +/- 0.004
saxs-regression-on-chili-3kGCN#2MSE: 0.008 +/- 0.000
saxs-regression-on-chili-3kGraphSAGE#3MSE: 0.008 +/- 0.001
saxs-regression-on-chili-3kGAT#4MSE: 0.008 +/- 0.000
saxs-regression-on-chili-3kGraphUNet#5MSE: 0.008 +/- 0.000
saxs-regression-on-chili-3kGIN#6MSE: 0.008 +/- 0.000
saxs-regression-on-chili-3kPMLP#7MSE: 0.022 +/- 0.025
saxs-regression-on-chili-3kMean#8MSE: 0.037
space-group-classification-on-chili-100kEdgeCNN#1F1-score (Weighted): 0.158 +/- 0.035
space-group-classification-on-chili-100kPMLP#2F1-score (Weighted): 0.047 +/- 0.012
space-group-classification-on-chili-100kGraphSAGE#3F1-score (Weighted): 0.044 +/- 0.002
space-group-classification-on-chili-100kGAT#4F1-score (Weighted): 0.044 +/- 0.001
space-group-classification-on-chili-100kGCN#5F1-score (Weighted): 0.043 +/- 0.001
space-group-classification-on-chili-100kGraphUNet#6F1-score (Weighted): 0.043 +/- 0.000
space-group-classification-on-chili-100kGIN#7F1-score (Weighted): 0.043 +/- 0.000
space-group-classification-on-chili-100kMost Frequent Class#8F1-score (Weighted): 0.010
space-group-classification-on-chili-100kRandom#9F1-score (Weighted): 0.002 +/- 0.001
space-group-classification-on-chili-3kEdgeCNN#1F1-score (Weighted): 0.733 +/- 0.207
space-group-classification-on-chili-3kGraphSAGE#2F1-score (Weighted): 0.151 +/- 0.045
space-group-classification-on-chili-3kPMLP#3F1-score (Weighted): 0.135 +/- 0.006
space-group-classification-on-chili-3kGIN#4F1-score (Weighted): 0.125 +/- 0.026
space-group-classification-on-chili-3kGAT#5F1-score (Weighted): 0.113 +/- 0.013
space-group-classification-on-chili-3kMost Frequent Class#6F1-score (Weighted): 0.108
space-group-classification-on-chili-3kGCN#7F1-score (Weighted): 0.099 +/- 0.019
space-group-classification-on-chili-3kGraphUNet#8F1-score (Weighted): 0.095 +/- 0.036
space-group-classification-on-chili-3kRandom#9F1-score (Weighted): 0.009 +/- 0.008
x-ray-pdf-regression-on-chili-100kMean#1MSE: 0.007
x-ray-pdf-regression-on-chili-100kEdgeCNN#2MSE: 0.012 +/- 0.000
x-ray-pdf-regression-on-chili-100kPMLP#3MSE: 0.013 +/- 0.000
x-ray-pdf-regression-on-chili-100kGAT#4MSE: 0.013 +/- 0.000
x-ray-pdf-regression-on-chili-100kGraphUNet#5MSE: 0.013 +/- 0.000
x-ray-pdf-regression-on-chili-100kGIN#6MSE: 0.013 +/- 0.000
x-ray-pdf-regression-on-chili-100kGCN#7MSE: 0.014 +/- 0.000
x-ray-pdf-regression-on-chili-100kGraphSAGE#8MSE: 0.037 +/- 0.026
x-ray-pdf-regression-on-chili-3kMean#1MSE: 0.008
x-ray-pdf-regression-on-chili-3kEdgeCNN#2MSE: 0.011 +/- 0.000
x-ray-pdf-regression-on-chili-3kGCN#3MSE: 0.012 +/- 0.000
x-ray-pdf-regression-on-chili-3kPMLP#4MSE: 0.012 +/- 0.000
x-ray-pdf-regression-on-chili-3kGraphSAGE#5MSE: 0.012 +/- 0.000
x-ray-pdf-regression-on-chili-3kGraphUNet#6MSE: 0.012 +/- 0.000
x-ray-pdf-regression-on-chili-3kGAT#7MSE: 0.029 +/- 0.030
xrd-regression-on-chili-100kEdgeCNN#1MSE: 0.006 +/- 0.000
xrd-regression-on-chili-100kPMLP#2MSE: 0.008 +/- 0.001
xrd-regression-on-chili-100kGCN#3MSE: 0.009 +/- 0.000
xrd-regression-on-chili-100kGraphUNet#4MSE: 0.009 +/- 0.000
xrd-regression-on-chili-100kGIN#5MSE: 0.009 +/- 0.000
xrd-regression-on-chili-100kGraphSAGE#6MSE: 0.018 +/- 0.014
xrd-regression-on-chili-100kMean#7MSE: 0.021
xrd-regression-on-chili-100kGAT#8MSE: 0.108 +/- 0.172
xrd-regression-on-chili-3kEdgeCNN#1MSE: 0.008 +/- 0.001
xrd-regression-on-chili-3kGCN#2MSE: 0.010 +/- 0.000
xrd-regression-on-chili-3kPMLP#3MSE: 0.010 +/- 0.000
xrd-regression-on-chili-3kGraphSAGE#4MSE: 0.010 +/- 0.000
xrd-regression-on-chili-3kGAT#5MSE: 0.010 +/- 0.000
xrd-regression-on-chili-3kGraphUNet#6MSE: 0.010 +/- 0.000
xrd-regression-on-chili-3kMean#7MSE: 0.017

Rank counts only results with a code link.