Media Effects Research Lab - Research Archive
Man versus Machine
Student Researcher(s)
Andrew J. Gambino (Ph.D Candidate);
Jinyoung Kim (Ph.D Candidate);
Faculty Supervisor
Introduction
Modern robots that are equipped with higher-level skills to conduct various tasks, have taken over many tasks that are traditionally performed by humans (Nass, Lombard, Henriksen, & Steuer, 1995). In fact, many robots these days have sophisticated senses and skills and can undertake duties that have been associated with humanness, such as greeting customers at department stores, depositing or withdrawing money out of bank accounts, and even conducting complex surgery. For instance, Toshiba Corporation recently placed a humanoid robot at the information desk of a department store in Tokyo to provide customer service for its customers (Hongo, 2015). However, it is no longer the case that robots are only conducting repetitive work that can be easily pre-programmed. One such area is that of AI-writers or computational journalism. Because writing is thought of as a very human task, the emergence of AI-writers has been met with a wary eye (Morozov, 2012). In a sense, it has become clear that it may be robots, not monkeys, who might be writing the next Shakespeare.
Because this technology is new, research on the topic of AI-writers is slim. However, two studies have explored the phenomenon. Kaa and Krahmer (2014) used the labels of “computer” and “journalist” to test reader effects on source and message credibility, and found that there were no significant differences, except in the case of the participant being a journalist, in which case the journalist was perceived as more trustworthy. Similarly, Edwards, Edwards, Spence, and Shelton (2014) found that twitter “bot” agents did not differ from the human agent in measures of source credibility or communication competence.
The aforementioned studies suggest the similarity in reactions to AI and human sources, but we believed a more rigorous theoretical approach was necessary to understand (1) if there are differential effects of an AI-writer as source, and (2) the psychological mechanisms through which such effects may occur. Therefore, we framed this research using Sundar’s (2008) MAIN model, which suggests the heuristic processing of content, and hypothesizes on the effects of source interface cues. We utilized the past research on source-orientation (Sundar & Nass, 2001) and the uncanny valley (Mori, 1972) to inform the current study.
Research Questions
RQ1: What is the relationship between news writer, authority cue, and topic, on credibility and behavioral intentions?
RQ2: Will this relationship be mediated by eeriness or novelty?
H1: Higher level of authority cue will lead to higher credibility judgements and behavioral intentions.
Method
A 2 (source: AI vs. Human) x 2 (authority: low vs. high) x 3 (genre: Health, Politics, Finance) + 3 (control for each genre) between-subjects online experiment was conducted to test the hypothesis and research questions. Participants (N = 435) were randomly assigned to one of the 15 conditions, in which they were exposed to a news-story stimuli which varied on the aforementioned factors, and subsequently filled out a questionnaire regarding their perceptions of the credibility of the source, message, website, as well as their behavioral intentions.
Stimuli
Results
In summary, we found that there was a significant interaction effect between news writer and topic, as well as a generally positive effect for high authority. Specifically, the human writer was preferred in health news, but the AI-writer was preferred in financial news. There were no significant interactions found between news writer and authority cue, nor were there any significant three-way interactions on the dependent variables. Furthermore, we found that in health news, paths through eeriness, eeriness and writer trust, and novelty mediated the news writer effect. The authority cue positive effect was relatively consistent in the predicted direction.
Conclusion
The increased agency being given to machines may be a boon for many, but those who think it will always be accepted by the public may need to take a step back and research the psychological effects that emerge in the human-AI relationship. This study suggests that the machine may have a distinct psychological persona, and that individuals may even prefer machines as writers in some contexts, but may also be uncomfortable with AIs writing health news. In addition, the study contributes to the literature of source-orientation in digital contexts. In this case, the effect of the news-writer, perhaps because of its novelty, was not moderated by that of the gatekeeper or editorial outlet. Furthermore, this study provides evidence that perceived eeriness may be a key psychological variable to consider in future HRI / HCI studies, even in the absence of an embodied agent.
In closing, we will address a question posed by Levy (2012) in his wired story on Narrative Science. In the piece, Levy asked if an algorithm could write a better news story than a human. Of course, there is no perfect way to measure what a better news story is, but our study answers this question from the perspective of the newsreader. Finding that at least we think an algorithm can write a better news story than a human, in some contexts.
For more details regarding the study contact
Dr. S. Shyam Sundar by e-mail at sss12@psu.edu or by telephone at (814) 865-2173