AI News

The paper introduces GeoSpatialBench, a benchmark for testing whether multimodal large language models can understand and actively modify spatial configurations under explicit geometric constraints. It targets a limitation in earlier evaluations, which have focused mainly on static question answering over small or synthetic scenes. The benchmark covers an escalating set of spatial cognition tasks, from recognizing spatial relations and performing geometric reasoning to constrained spatial editing. It uses real-world urban data from six geographically diverse cities, with 28 topological, directional, metric, and ordering relations represented through structured GeoJSON and map-image modalities. Ground-truth references and editing constraints are verified with GIS operations. In evaluations of frontier multimodal models, structured GeoJSON consistently produced better results than map images, while combining both modalities did not provide a reliable improvement over GeoJSON alone. Models handled qualitative topological and ordering relations relatively well, but their performance declined on quantitative metric relations. Editing ordering relations was harder than answering questions about them, and most failed edits violated at least one required spatial constraint. The authors characterize these findings as a gap between passively recognizing spatial relations and actively manipulating geometric configurations. They present GeoSpatialBench as an open diagnostic testbed for studying the spatial limitations of current models and for future work on physically grounded world models and embodied spatial AI.