The System is Down

Recently, AI created by Facebook had to be shut down because it developed its own language:

An artificial intelligence system being developed at Facebook has created its own language. It developed a system of code words to make communication more efficient. The researchers shut the system down as it prompted concerns we could lose control of AI.

The observations made at Facebook are the latest in a long line of similar cases. In each instance, an AI being monitored by humans has diverged from its training in English to develop its own language. The resulting phrases appear to be nonsensical gibberish to humans but contain semantic meaning when interpreted by AI "agents."

In one exchange illustrated by the company, the two negotiating bots, named Bob and Alice, used their own language to complete their exchange. Bob started by saying "I can i i everything else," to which Alice responded "balls have zero to me to me to me…" The rest of the conversation was formed from variations of these sentences.

While it appears to be nonsense, the repetition of phrases like "i" and "to me" reflect how the AI operates. The researchers believe it shows the two bots working out how many of each item they should take. Bob's later statements, such as "i i can i i i everything else," indicate how it was using language to offer more items to Alice. When interpreted like this, the phrases appear more logical than comparable English phrases like "I'll have three and you have everything else."

The reasoning behind the "fear of losing control" is wanting to make sure the process undertaken by the AI can be understood by humans, if necessary. But the reason we develop and use AI is specifically to do things that humans can't, or that it would take humans a long time to do manually. While I can understand the need for monitoring, even if the AI bots were speaking English, it would probably still take a while to dig in and find out what they're doing. The concept of AI really seems to be about the need to put difficult processes into a black box.

So, alas, Bob and Alice (and all of their friends) are now offline.