This dissertation project, guided by the cognitive theory of multimedia learning (CTML), explores how popular constructions of news media, such as those utilizing text, images, and videos, affect one’s memory for disinformation. In particular, this investigation scrutinizes the individuals’ capacity to remember story details from both factual and disinformation news following exposure to messages presented in different modality formats. To this end, two online experimental studies were conducted using manipulated mock news articles surrounding political events in the U.S. The first study (n = 569), utilizing articles on the 2021 U.S. Capitol riot, found that text and image articles facilitated participants’ memory for the factual news condition better than videos. Conversely, participants who read a text article or watched a news video performed significantly better than those exposed to image articles in the disinformation condition. Such findings indicate that additional factors, such as expectancy violation and the construction of the news play an important role in remembering political news. Moreover, results also indicate that partisanship is closely related to the retention of story details, although this should not be considered an indicator that one party remembers disinformation more than the other.

Building on the findings from the first study, the second study probed further into the “black box” relationship between modality and retention. As visual elements are uniquely situated to evoke emotional responses from viewers of political news, the second study (n = 923) expanded the CTML by incorporating an affective dimension into the modality and multimedia principles to account for the potential mediating role of discrete emotions. Lastly, as attitudinal congruence of the news article topic can lead to the arousal of both positive and negative emotions, the study included news that either aligned or violated party consistent attitudes. Firstly, the findings from Study 2 suggest that the construction of the news content plays an important role in the cognitive processing of the different modalities. For instance, the addition of headlines may have reduced the effects of visual priming cues, which allowed for the image modality to be effective in aiding memory formation across both factual and disinformation news content. Moreover, the utilization of novel disinformation, which is arguably the most frequently occurring type of purposefully misleading information found online, decreased the potential for expectancy violation. Aside from modality, emotion was also found to be a predictor of content memory, indicating that the arousal of anger and enthusiasm, not anxiety, plays a role in the number of story details that individuals retain.

Interestingly, Republicans and Democrats not only had differentiating responses to pro- and counter-attitudinal news content, affecting the strength of emotional arousal, but partisans also did not process visual modalities the same either. Thus, suggesting that the information environment plays a pivotal role in how partisans respond to political news. Lastly, although emotion was situated as a mediator between exposure to modality and content memory, findings do not support this notion. Instead, the results suggest complementary processing routes to memory formation, namely through cognitive (i.e., modality) and affective (i.e., discrete emotions) pathways. Therefore, findings lend support for the notion that it is not always the sensational visual, ingrained with compelling negative or positive imagery, that enhances memory – neutral constructions of disinformation news made to mirror credible news reporting can also be stored into memory.

Findings from both studies contribute to the advancement of the CTML in its application to learning from political news as well as the role of emotions in the retention of disinformation story details. For instance, it was found that the effects of different visual modalities and the strength of discrete emotional arousals on one’s memory for disinformation may vary depending on individuals’ partisanship, the salience of the news event, as well as the underlying information environment.