Multiphase flows in porous media are important in many natural and industrial processes. Pore-scale models for multiphase flows have seen rapid development in recent years and are becoming increasingly useful as predictive tools in both academic and industrial applications. However, quantitative comparisons between different pore-scale models, and between these models and experimental data, are lacking. Here, we perform an objective comparison of a variety of state-of-the-art pore-scale models, including lattice Boltzmann, stochastic rotation dynamics, volume-of-fluid, level-set, phase-field, and pore-network models. As the basis for this comparison, we use a dataset from recent microfluidic experiments with precisely controlled pore geometry and wettability conditions, which offers an unprecedented benchmarking opportunity. We compare the results of the 14 participating teams both qualitatively and quantitatively using several standard metrics, such as fractal dimension, finger width, and displacement efficiency. We find that no single method excels across all conditions and that thin films and corner flow present substantial modeling and computational challenges.