MassSpecGym: A benchmark for the discovery and identification of molecules

Benchmark Model Rank Results
de-novo-molecule-generation-from-ms-msRandom chemical generation#1Top-1 Accuracy: 0.00Top-1 MCES: 28.59Top-1 Tanimoto: 0.07
de-novo-molecule-generation-from-ms-msSELFIES Transformer#2Top-1 Accuracy: 0.00Top-1 MCES: 33.28Top-1 Tanimoto: 0.10
de-novo-molecule-generation-from-ms-msSMILES Transformer#3Top-1 Accuracy: 0.00Top-1 MCES: 53.80Top-1 Tanimoto: 0.07
de-novo-molecule-generation-from-ms-ms-1Random chemical generation#3Top-1 Accuracy: 0.00Top-1 MCES: 21.11Top-1 Tanimoto: 0.08
de-novo-molecule-generation-from-ms-ms-1SELFIES Transformer#4Top-1 Accuracy: 0.00Top-1 MCES: 38.88Top-1 Tanimoto: 0.08
de-novo-molecule-generation-from-ms-ms-1SMILES Transformer#5Top-1 Accuracy: 0.00Top-1 MCES: 79.39Top-1 Tanimoto: 0.03
molecule-retrieval-from-ms-ms-spectrum-bonusMIST#1Hit rate @ 1: 9.57Hit rate @ 5: 22.11Hit rate @ 20: 41.12
molecule-retrieval-from-ms-ms-spectrum-bonusDeepSets + Fourier features#2Hit rate @ 1: 6.56Hit rate @ 5: 16.46Hit rate @ 20: 33.46
molecule-retrieval-from-ms-ms-spectrum-bonusFingerprint FFN#3Hit rate @ 1: 5.09Hit rate @ 5: 14.69Hit rate @ 20: 31.97
molecule-retrieval-from-ms-ms-spectrum-bonusDeepSets#4Hit rate @ 1: 4.42Hit rate @ 5: 14.46Hit rate @ 20: 30.76
molecule-retrieval-from-ms-ms-spectrum-bonusRandom#5Hit rate @ 1: 3.06Hit rate @ 5: 11.35Hit rate @ 20: 27.74
molecule-retrieval-from-ms-ms-spectrum-on-massspecgymMIST#1Hit rate @ 1: 14.64Hit rate @ 5: 34.87Hit rate @ 20: 59.15
molecule-retrieval-from-ms-ms-spectrum-on-massspecgymDeepSets + Fourier features#2Hit rate @ 1: 5.24Hit rate @ 5: 12.58Hit rate @ 20: 28.21
molecule-retrieval-from-ms-ms-spectrum-on-massspecgymFingerprint FFN#3Hit rate @ 1: 2.54Hit rate @ 5: 7.59Hit rate @ 20: 20.00
molecule-retrieval-from-ms-ms-spectrum-on-massspecgymDeepSets#4Hit rate @ 1: 1.47Hit rate @ 5: 6.21Hit rate @ 20: 19.23
molecule-retrieval-from-ms-ms-spectrum-on-massspecgymRandom#5Hit rate @ 1: 0.37Hit rate @ 5: 2.01Hit rate @ 20: 8.22
ms-ms-spectrum-simulation-bonus-chemicalFraGNNet#1Hit Rate @ 1: 31.93Hit Rate @ 5: 63.20Hit Rate @ 20: 82.70
ms-ms-spectrum-simulation-bonus-chemicalFFN Fingerprint#2Hit Rate @ 1: 7.62Hit Rate @ 5: 22.70Hit Rate @ 20: 44.12
ms-ms-spectrum-simulation-bonus-chemicalGNN#3Hit Rate @ 1: 3.63Hit Rate @ 5: 13.55Hit Rate @ 20: 33.77
ms-ms-spectrum-simulation-bonus-chemicalPrecursor m/z#4Hit Rate @ 1: 2.09Hit Rate @ 5: 8.52Hit Rate @ 20: 22.65
ms-ms-spectrum-simulation-on-massspecgymFraGNNet#1Cosine Similarity: 0.52Jensen-Shannon Similarity: 0.47
ms-ms-spectrum-simulation-on-massspecgymFFN Fingerprint#2Cosine Similarity: 0.25Jensen-Shannon Similarity: 0.24
ms-ms-spectrum-simulation-on-massspecgymGNN#3Cosine Similarity: 0.19Jensen-Shannon Similarity: 0.20
ms-ms-spectrum-simulation-on-massspecgymPrecursor m/z#4Cosine Similarity: 0.15Jensen-Shannon Similarity: 0.15