Improving Generalization in Federated Learning by Seeking Flat Minima

Benchmark Model Rank Results
federated-learning-on-cifar-100-alpha-0-10FedASAM + SWA#1ACC@1-100Clients: 42.64
federated-learning-on-cifar-100-alpha-0-10FedASAM#2ACC@1-100Clients: 39.76
federated-learning-on-cifar-100-alpha-0-10FedSAM + SWA#3ACC@1-100Clients: 39.51
federated-learning-on-cifar-100-alpha-0-10FedSAM#4ACC@1-100Clients: 36.93
federated-learning-on-cifar-100-alpha-0-10FedAvg#5ACC@1-100Clients: 36.74
federated-learning-on-cifar-100-alpha-0-20FedASAM + SWA#1ACC@1-100Clients: 41.62
federated-learning-on-cifar-100-alpha-0-20FedASAM#2ACC@1-100Clients: 40.81
federated-learning-on-cifar-100-alpha-0-20FedSAM + SWA#3ACC@1-100Clients: 39.24
federated-learning-on-cifar-100-alpha-0-20FedAvg#4ACC@1-100Clients: 38.59
federated-learning-on-cifar-100-alpha-0-20FedSAM#5ACC@1-100Clients: 38.56
federated-learning-on-cifar-100-alpha-0-5FedASAM + SWA#1ACC@1-100Clients: 42.01
federated-learning-on-cifar-100-alpha-0-5FedSAM + SWA#2ACC@1-100Clients: 39.3
federated-learning-on-cifar-100-alpha-0-5FedASAM#3ACC@1-100Clients: 36.04
federated-learning-on-cifar-100-alpha-0-5FedSAM#4ACC@1-100Clients: 31.04
federated-learning-on-cifar-100-alpha-0-5FedAvg#5ACC@1-100Clients: 30.25
federated-learning-on-cifar-100-alpha-0-5-10FedASAM + SWA#1ACC@1-100Clients: 48.72
federated-learning-on-cifar-100-alpha-0-5-10FedSAM + SWA#2ACC@1-100Clients: 46.76
federated-learning-on-cifar-100-alpha-0-5-10FedASAM#3ACC@1-100Clients: 46.58
federated-learning-on-cifar-100-alpha-0-5-10FedSAM#4ACC@1-100Clients: 44.84
federated-learning-on-cifar-100-alpha-0-5-10FedAvg#5ACC@1-100Clients: 41.27
federated-learning-on-cifar-100-alpha-0-5-20FedASAM + SWA#1ACC@1-100Clients: 48.27
federated-learning-on-cifar-100-alpha-0-5-20FedASAM#2ACC@1-100Clients: 47.78
federated-learning-on-cifar-100-alpha-0-5-20FedSAM + SWA#3ACC@1-100Clients: 46.47
federated-learning-on-cifar-100-alpha-0-5-20FedSAM#4ACC@1-100Clients: 46.05
federated-learning-on-cifar-100-alpha-0-5-20FedAvg#5ACC@1-100Clients: 42.17
federated-learning-on-cifar-100-alpha-0-5-5FedASAM + SWA#1ACC@1-100Clients: 49.17
federated-learning-on-cifar-100-alpha-0-5-5FedSAM + SWA#2ACC@1-100Clients: 47.96
federated-learning-on-cifar-100-alpha-0-5-5FedASAM#3ACC@1-100Clients: 45.61
federated-learning-on-cifar-100-alpha-0-5-5FedSAM#4ACC@1-100Clients: 44.73
federated-learning-on-cifar-100-alpha-0-5-5FedAvg#5ACC@1-100Clients: 40.43
federated-learning-on-cifar-100-alpha-1000-10FedASAM#1ACC@1-100Clients: 54.97
federated-learning-on-cifar-100-alpha-1000-10FedASAM + SWA#2ACC@1-100Clients: 54.79
federated-learning-on-cifar-100-alpha-1000-10FedSAM + SWA#3ACC@1-100Clients: 53.67
federated-learning-on-cifar-100-alpha-1000-10FedSAM#4ACC@1-100Clients: 53.39
federated-learning-on-cifar-100-alpha-1000-10FedAvg#5ACC@1-100Clients: 50.25
federated-learning-on-cifar-100-alpha-1000-20FedASAM#1ACC@1-100Clients: 54.5
federated-learning-on-cifar-100-alpha-1000-20FedSAM + SWA#2ACC@1-100Clients: 54.36
federated-learning-on-cifar-100-alpha-1000-20FedASAM + SWA#3ACC@1-100Clients: 54.1
federated-learning-on-cifar-100-alpha-1000-20FedSAM#4ACC@1-100Clients: 53.97
federated-learning-on-cifar-100-alpha-1000-20FedAvg#5ACC@1-100Clients: 50.66
federated-learning-on-cifar-100-alpha-1000-5FedASAM#1ACC@1-100Clients: 54.81
federated-learning-on-cifar-100-alpha-1000-5FedSAM#2ACC@1-100Clients: 54.01
federated-learning-on-cifar-100-alpha-1000-5FedSAM + SWA#3ACC@1-100Clients: 53.9
federated-learning-on-cifar-100-alpha-1000-5FedASAM + SWA#4ACC@1-100Clients: 53.86
federated-learning-on-cifar-100-alpha-1000-5FedAvg#5ACC@1-100Clients: 49.92
federated-learning-on-cityscapesSiloBN + ASAM#1mIoU: 49.75
federated-learning-on-cityscapesSiloBN + SAM#2mIoU: 49.1
federated-learning-on-cityscapesSiloBN#3mIoU: 45.96
federated-learning-on-cityscapesFedSAM + SWA#4mIoU: 43.42
federated-learning-on-cityscapesFedASAM + SWA#5mIoU: 43.02
federated-learning-on-cityscapesFedAvg + SWA#6mIoU: 42.48
federated-learning-on-cityscapesFedASAM#7mIoU: 42.27
federated-learning-on-cityscapesFedSAM#8mIoU: 41.22
federated-learning-on-cityscapesFedAvg#9mIoU: 38.65
federated-learning-on-landmarks-user-160kFedASAM + SWA#1Acc@1-1262Clients: 68.32
federated-learning-on-landmarks-user-160kFedSAM + SWA#2Acc@1-1262Clients: 68.12
federated-learning-on-landmarks-user-160kFedAvg + SWA#3Acc@1-1262Clients: 67.52
federated-learning-on-landmarks-user-160kFedASAM#4Acc@1-1262Clients: 64.23
federated-learning-on-landmarks-user-160kFedSAM#5Acc@1-1262Clients: 63.72
federated-learning-on-landmarks-user-160kFedAvg#6Acc@1-1262Clients: 61.91