In a post last semester, we used a digital archive to create new objects by designing a little tweeting machine. This technique – using an archive to create an “intelligent” machine – is not only useful for such coding exercises or publicity stunts like the Next Rembrandt. These computational methods are also at the heart of one of Silicon Valley’s trendiest technologies: the interactive “chat bot”.
Everyone does bots – short for “robots” – now: at its most recent developer conference, Facebook announced Messenger Bots for interaction through the Facebook Messenger service; Microsoft is integrating chat bots into Skype; the messenger service Kik is starting a “Build your bot” campaign and its competitor Telegram is offering $ 1 million in grants to bot developers. “Right now”, the Wall Street Journal writes, “Silicon Valley’s hopes seem to be pinned on the prospect of a bot revolution”.
These bots are pieces of software that interact with their users: they answer questions, help with online tasks, conduct business transactions or simply entertain. They can do so over any chat interface or text messaging service.
Their algorithms have been trained on an archive of digital data of some sorts. That’s how they learn to construct sentences and how to interact with users. They start by analyzing large data sets of the kind of interactions they will be dealing with, then, based on this data, they do their own thing.
That can – and will – go wrong. Sometimes in obvious, but always in less obvious ways.
Exhibit A of an obvious failure is Microsoft’s Twitter chat bot Tay. Started last month and intended to converse on Twitter in the style of an American teenager, Tay instead quickly learned how to be racist and started calling other people names. Microsoft silenced it after less than a week.
Which gets us to the main issue to be addressed here: digital archives and their relation to the “Uncanny Valley“. This not-quite-technical term (illustrated here) refers to the “uncanny” borderlands between between humans and robots: the strange region inhabited by machines that look or act incredibly human, but are not quite there.
We real people might not be able to pinpoint why they don’t pass the test. They certainly don’t, but they do come close, and we tend to find that disconcerting or uncanny. Examples of inhabitants of the uncanny valley include sophisticated humanoid robots like the one pictured above.
Sophisticated chat bots have the same potential for creepiness. Talking is something we associate very strongly with being human. (when writing about Microsoft’s Tay above, for instance, I wrote “Microsoft silenced her after less than a week” before changing the pronoun to it – a good number of readers would probably not have stumbled over either).
When bots talk to us, we thus instinctively treat them like human interlocutors. For instance, we try to make sense of whatever garbled sentences they produce, just like we do with our fellow humans. And it is exactly when we think we “understand” them that the creepiness comes in: the machine forms sentences, but we realize it doesn’t know what they mean. The shape of the sentence is there, but not its soul. And that is the language of the uncanny valley.
Thus, Tay tweeted:“ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism” or, when asked about the Holocaust, opined that “it was made up 👏”. Thus, my own little sentence generator suddenly informed me that it was ” Devising death .”
These machines don’t mean any harm. They just combine words according to a probabilistic model. But since they have the grammatical skills, their messages can be disconcerting or creepy. How creepy and how disconcerting depends on the data set, the archive we are training them on: the more data you have, the more human-like the bot becomes. The more data you have, the harder it is to curate your archive and avoid unforeseen behavior. Etc.
After all, these machines don’t say or do things that humans wouldn’t or couldn’t do. The difference is that they have no understanding of context or meaning. And as long as they don’t, they will have to be staying in uncanny valley.
Image Credit: Repliee Q2. Photo by Max Braun https://www.flickr.com/photos/72645106@N00/1489103461. Distributed under a Creative Commons Share-Alike License: https://creativecommons.org/licenses/by-sa/2.0/legalcode. Not modified.