Self-doubting robots override humans’ wishes

Evidence that suggests making robots less self-assured might make them easier to integrate into society, a team of researchers from the University of California at Berkeley has found. In their paper uploaded to the prepress server arXiv, the group explained their theory and the results of a simulation they ran to test it.

As artificial intelligence and robot technology improve, we humans are going to be faced with some decisions—one, the authors suggest, could be how much self-confidence to give them. Too much, and they might try to override our wishes; too little might make them less than useful.

To better understand the problem, the researchers created a mathematical model meant to depict an interaction between a robot and a human with parameters for adjusting self-confidence. In one simulation, a robot with a built-in off switch was asked to perform a desired task. At that point, a human was given the option of allowing the robot to continue or hit its off switch. But the robot had the ability to override its own off switch, and thus the human’s wishes. The researchers found that robots given a healthy dose of self-confidence, as might be expected, chose to override the human’s wishes and turn themselves back on. When given just a little bit of self-confidence, on the other hand, the robot stayed off, even if it judged itself doing a good job.

In the real world, the researchers offer an example of what we humans already face—the newsfeed on Facebook. What started out as a means of offering news that a bot thought would be useful has become a major nuisance, because it instead offers a constant stream of fake news. This could have been averted, the researchers contend, if the bot had less confidence.

On the other hand, the researchers note, we have to be careful to not take away too much confidence—a robot might need to override a child’s wishes, for example, when they are challenged to stop or change routes. One way to give robots the right amount of self-confidence, they suggest, might be to give them more information to work with. One example would be a robot that makes coffee only in the morning, because it has learned that is when coffee is wanted.