Researchers from the University of Glasgow and Florida State University have developed a new framework that improves the accuracy and scalability of an AI foundation model when analysing geospatial data.
Researchers from the University of Glasgow and Florida State University have developed a new approach that significantly improves the performance of an AI foundation model for analysing geospatial data, potentially benefiting applications ranging from public health and housing to climate change and urban planning.
The research, published in the International Journal of Geographical Information Science, enhances TabPFN, a machine learning foundation model designed to analyse tabular data such as spreadsheets and databases. The team found that while TabPFN performs well on many geospatial tasks, it struggles with larger datasets and with capturing relationships between nearby locations.
To address these limitations, the researchers developed an open-source framework called TabPFN-GSA, based on a new methodology known as Geospatial Sparse Attention (GSA). The approach gives the model a better understanding of geographical relationships by directing greater attention to nearby data points while still considering relevant information from further away.
Dr Mingshu Wang, of the University of Glasgow’s School of Geographical & Earth Sciences, said: “The first law of geography is that ‘everything is related to everything else, but near things are more related than distant things’. In geospatial data, that means that we can scrutinise how closely data points are related to each other in space in order to find connections and draw conclusions.
“General-purpose tabular models can be very powerful, but they are trained to treat rows as independent observations – they don’t automatically understand the principles of geospatial data. That’s why we set out to expand TabPFN’s ability to make the connections between tabulated geospatial data instead of trying to build and train a new model from scratch.”
The team tested TabPFN-GSA on 30 synthetic datasets and four real-world datasets covering air pollution, housing prices, poverty and the 2020 US presidential election. The enhanced model consistently delivered more accurate predictions and successfully processed a 70,000-row dataset that the original model was unable to analyse.
University of Glasgow PhD student Rui Deng, the paper’s first author, said: “In geospatial data, each row of the table has its own locational information like map coordinates. In our Geospatial Sparse Attention model, we divide the whole region covered by the table into a grid, so we know the relative distance between all the data points. Then, we guide the model to attend more to nearer points rather than distant ones, focusing it on the local context. We didn’t modify TabPFN itself; instead, we provided it with a better context to improve the model’s performance.”
The researchers believe the open-source technology could be used by universities, government agencies, local authorities and data analytics organisations, while allowing sensitive geospatial data to be processed securely on local computers without relying on cloud-based AI services.
