Students research AI and political discourse in Project Frozone

A look at the headlines in the news, or even just scrolling our personal social media feeds, it is evident that online political discussions are fraught with problems from misinformation to toxic exchanges between users.  For the past two semesters, students Garrett McKenzie (Data Science and Computer Science ’27), Laura Rider (Engligh Lit and Computer Science ’26), and Bethanie Hackett (Computer Science and Political Science) have worked with Professor of Computer Science Dr. Stephen Davies to test and an important question: can Artificial Intelligence be used to identify and “cool” the temperature of such exchanges?

Project Frozone is a two-semester research project in computational social science that examines how AI can be used to cool (pun intended) the temperature of online political discourse. The AI works by first scanning chatrooms and detecting posts that contain misinformation, misrepresentation, toxicity, fallacious reasoning, gender bias, or racial bias. Once this “cooling” AI has detected one of these unproductive aspects, it generates a response that is intended to neutralize the effects of the unproductive dialogue and moderate the conversation. This response methodology is called counter speech and is a relatively new topic in natural language processing (NLP).

An experiment was conducted in the Fall 2025 semester to examine the effectiveness of the AI, which had UMW student volunteer participants complete surveys and interact with the cooling AI named “FroBot” in a private chatroom with an initial political topic given as a prompt. Also present in the chat room were the “HotBot” AI and the “CoolBot” AI, intended to create unproductive posts and neutral posts, respectively. All three of these bots were built off Gemini2.5-Flash through VertexAI and taught on a mix of handmade and collected data. The experiment found that the “FroBot” was moderately effective in helping to cool the conversation when the “HotBot” made a post that was inflammatory. However, participants reported several instances of the AIs going off script by generating long paragraphs, odd sentences, and text announcing that they were AI.

Analysis of these results and possible continued experiments are planned for the Spring 2026 semester. Also planned for the Spring 2026 semester is a simulation-based experiment that will observe the effectiveness of the “FroBot” in a large-scale agent-based modeling (ABM) system. This system will use many AIs, each trained to mimic real-life social media users to simulate online interactions. The “FroBot” will be deployed into this simulation to see how it affects the conversations between the AI agents of the model.

Learning how to fine-tune and prompt engineer large-language models through this project has been a blast. Further, working with the research team has been a great experience and has taught me a lot about effective teamwork. (Garrett McKenzie)”

This project has provided me the opportunity to explore the field of data science in greater depth by working with large language models, particularly through examining the fine-tuning process and evaluating the many considerations involved in creating effective training datasets. (Bethanie Hackett)

I enjoyed seeing the different aspects of our project that we worked on throughout the semester come together while running the experiment sessions, including our work fine-tuning the bots, creating the chatroom and database interfaces, and designing the experiment itself. (Laura Rider)

Bethanie, Laura, and Garrett have designed a sensational experimental, and have fine-tuned the Gemini AI model using the latest technologies and best practices. It’s pretty amazing how lifelike it seems when ‘chatting’ with it, and how good a job it does of refocusing unproductive conversation! I look forward to continue to working with these talented students on the data analysis and simulation phases of the project. (Dr. Stephen Davies)

This research was presented at the Network for Undergraduate Research in Virginia Conference hosted at the University of Mary Washington in November, 2025. The students and Dr. Davies are working to produce a published paper to submit for potential publication in a peer-reviewed journal.