WebLINX: Real-World Website Navigation with Multi-Turn Dialogue

Benchmark Model Rank Results
conversational-web-navigation-on-weblinxLlama-2-13B#1Overall score: 25.21Intent Match: 81.91Element (IoU): 22.82…
conversational-web-navigation-on-weblinxS-LLaMA-2.7B#2Overall score: 25.02Intent Match: 84.00Element (IoU): 22.60…
conversational-web-navigation-on-weblinxLlama-2-7B#3Overall score: 24.57Intent Match: 82.64Element (IoU): 22.26…
conversational-web-navigation-on-weblinxFlan-T5-3B#4Overall score: 23.77Intent Match: 81.14Element (IoU): 20.31…
conversational-web-navigation-on-weblinxS-LLaMA-1.3B#5Overall score: 23.73Intent Match: 83.32Element (IoU): 20.54…
conversational-web-navigation-on-weblinxGPT-3.5F#6Overall score: 21.22Intent Match: 77.56Element (IoU): 18.64…
conversational-web-navigation-on-weblinxMindAct-3B#7Overall score: 20.94Intent Match: 79.89Element (IoU): 16.50…
conversational-web-navigation-on-weblinxFuyu-8B#8Overall score: 19.97Intent Match: 80.07Element (IoU): 15.70…
conversational-web-navigation-on-weblinxFlan-T5-780M#9Overall score: 17.27Intent Match: 80.02Element (IoU): 15.36…
conversational-web-navigation-on-weblinxPix2Act-1.3B#10Overall score: 16.88Intent Match: 81.80Element (IoU): 8.28…
conversational-web-navigation-on-weblinxMindAct-780M#11Overall score: 15.13Intent Match: 75.87Element (IoU): 13.39…
conversational-web-navigation-on-weblinxFlan-T5-250M#12Overall score: 14.99Intent Match: 79.69Element (IoU): 14.86…
conversational-web-navigation-on-weblinxMindAct-250M#13Overall score: 12.63Intent Match: 74.25Element (IoU): 12.05…
conversational-web-navigation-on-weblinxPix2Act-282M#14Overall score: 12.51Intent Match: 79.71Element (IoU): 6.20…
conversational-web-navigation-on-weblinxGPT-4T (Zero-Shot)#15Overall score: 10.72Intent Match: 41.66Element (IoU): 10.85…
conversational-web-navigation-on-weblinxGPT-4V (Zero-Shot)#16Overall score: 10.45Intent Match: 42.36Element (IoU): 10.91…
conversational-web-navigation-on-weblinxGPT-3.5T (Zero-Shot)#17Overall score: 8.51Intent Match: 42.77Element (IoU): 8.62…