CLIMATE IN THE MACHINE: From Risk Discrepancy to Conceptual Representation in Large Language Models 

Abstract

Large language models (LLMs), trained on vast text corpora, can generate human-like text and enable interactive communication; however, they can also encode and amplify societal narratives. Addressing the potential for these models to misinform the public on critical issues like climate change, this dissertation presents two interconnected studies.

Study 1 investigates a systematic discrepancy in LLM-generated climate assessments, specifically, a tendency towards overestimation, which is an exaggeration of climate change impacts compared to expert consensus. Using non-parametric statistical methods, the study compares expert ratings from the Intergovernmental Panel on Climate Change 2023 Synthesis Report with responses from GPT-family LLMs. Results reveal systematic overestimation of climate change impacts by these LLMs, a discrepancy that is particularly pronounced when the models are prompted to adopt the role of a climate scientist. This finding underscores the critical need to align LLM-generated climate information with established scientific consensus.

Building on these findings, Study 2 introduces the Conceptual Orientation, Range, and Neighborhood (CORe) framework to analyze the underlying conceptual representations of LLMs by contrasting models that exhibit overestimation with a baseline model that does not. The CORe analysis utilizes word embeddings and computational methods to map LLM responses into a semantic space. Contrastive analysis reveals that models exhibiting overestimation consistently display a more risk-oriented conceptual orientation, a narrower conceptual range, and a more hierarchical conceptual neighborhood. This suggests a structural link between an LLM’s conceptual representations and its observed behavioral output.

This dissertation advances research at the critical intersection of AI capabilities and societal discourse. It demonstrates how LLMs can produce systematic discrepancies in vital areas like climate change, a distortion that aligns with core concerns in misinformation studies. Crucially, beyond identifying these inaccuracies, the work introduces a deeper diagnostic approach using the CORe framework to analyze how the structure of LLMs’ conceptual representations may predispose them toward generating misaligned ratings.

This work makes three significant contributions. First, it empirically establishes a systematic discrepancy, overestimation, in AI-driven climate assessments. Second, it provides a sociotechnical explanation for this phenomenon by integrating the Social Amplification of Risk Framework and Social Role Theory with AI analysis. Third, it introduces a novel analytical framework (CORe) and associated prompting methodology (SCDEA) for LLM conceptual representation analysis, building upon the theory of conceptual spaces.

Date of Award	18 Sept 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Jian Hua Jonathan ZHU (Supervisor) & Ye SUN (Co-supervisor)