Beyond Language Modeling: An Exploration of Multimodal Pretraining

Open paper

Benchmark	Model	Rank	Results
text-to-image-generation-on-geneval	MoE (Per-Modality Shared Expert)	–	Overall: 0.367