Twitter Data Analysis for Geospatial Insights

In the Spring semester of 2024, Bonsai ACG’s marketing and business team undertook a significant Twitter data analysis project in partnership with the University of Miami’s Business School. The project, funded with a $5,000 budget, aimed to analyze and visualize geospatial trends across over 700,000 tweets. This large-scale data analysis effort provided deep insights into social media behavior and trends, offering invaluable information for both academic research and business applications. Our team began by managing the budgeting, data collection, and technical infrastructure needed to handle this vast dataset. We utilized Twitter’s professional API to gather the data, which included several millions of data points such as tweet content, tweet metadata, and user metadata. One of the main challenges we faced was that a significant portion of tweets lacked explicit location data, which was crucial for the geographic analysis we were tasked with conducting. To overcome this, we implemented geolocation inference using the Pigeo repository. This tool allowed us to infer the location of tweets even when explicit geotags were missing. However, one of the key challenges we encountered was understanding the dependencies of this older tool and learning how to effectively run subprocesses to make it function within our modern data pipeline. This required extensive research and trial-and-error as our team delved into documentation and community forums to troubleshoot compatibility issues and ensure smooth integration. By analyzing tweet metadata, including language and timezone information, we were able to estimate geographic data, enabling a robust analysis of social media trends by region. The data processing and analysis phase was extensive, involving the cleaning, sorting, and visualization of the tweet data using Python. Our team conducted an in-depth examination of the error rates associated with geolocation and data collection methods, ensuring the integrity of our findings. The final product was a series of visualizations and reports that illuminated social media trends across various geographic regions, offering both academic researchers and business analysts a powerful tool for understanding the dynamics of online communication. This project tested Bonsai Analysts’ ability to manage large-scale data collection and analysis efforts.