ShapeLLM: Universal 3D Object Understanding for Embodied Interaction

Benchmark Model Rank Results
3d-question-answering-3d-qa-on-3d-mm-vetShapeLLM-13B#1Overall Accuracy: 53.1
3d-question-answering-3d-qa-on-3d-mm-vetShapeLLM-7B#2Overall Accuracy: 47.4
few-shot-3d-point-cloud-classification-on-modelnet40-10-way-20-shotReCon++#1Overall Accuracy: 96.5Standard Deviation: 3.0
few-shot-3d-point-cloud-classification-on-modelnet40-5-way-20-shotReCon++#1Overall Accuracy: 99.5Standard Deviation: 0.8
zero-shot-transfer-3d-point-cloud-classification-on-scanobjectnnReCon++#1OBJ_ONLY Accuracy(%): 65.4
3d-object-captioning-on-objaverseShapeLLM-13B#2GPT-4: 48.94Sentence-BERT: 48.52SimCSE: 49.98
3d-object-captioning-on-objaverseShapeLLM-7B#4GPT-4: 46.92Sentence-BERT: 48.20SimCSE: 49.23
3d-point-cloud-linear-classification-on-modelnet40ReCon++#2Overall Accuracy: 93.6
few-shot-3d-point-cloud-classification-on-modelnet40-10-way-10-shotReCon++#2Overall Accuracy: 94.5Standard Deviation: 4.1
few-shot-3d-point-cloud-classification-on-modelnet40-5-way-10-shotReCon++#2Overall Accuracy: 98.0Standard Deviation: 2.3
generative-3d-object-classification-on-modelnet40ShapeLLM-7B#2ModelNet40 (Average): 53.08
generative-3d-object-classification-on-modelnet40ShapeLLM-13B#3ModelNet40 (Average): 52.96
generative-3d-object-classification-on-objaverseShapeLLM-7B#2Objaverse (Average): 54.50
generative-3d-object-classification-on-objaverseShapeLLM-13B#4Objaverse (Average): 54.00
3d-point-cloud-classification-on-modelnet40ReCon++#3Overall Accuracy: 95.0
3d-point-cloud-classification-on-scanobjectnnReCon++#3Overall Accuracy: 95.25OBJ-BG (OA): 98.80OBJ-ONLY (OA): 97.59
zero-shot-transfer-3d-point-cloud-classification-on-modelnet40ReCon++#3Accuracy (%): 87.3