AiTech paper "Improving Confidence in the Estimation of Values and Norms" presented at the COINE Workshop @AAMAS 2020

The paper was authored by Luciano Cavalcante Siebert, Rijk Mercuur, Virginia Dingnum, Jeroen Van den Hoven and Catholijn Jonker. It is motivated by the growing need of aligning the behavior of autonomous agents with human moral values and norms. For this, an autonomous agent will need to understand to what extent a given set of values and norms are relevant and preferred in a context.

It studied the conditions under which such agents can make confident estimates of one’s preference on values and norms from observed behavior. The experiment was framed in terms of the ultimatum game to make it crisp and amenable to a sound conceptual and theoretical approach. The main contributions of this paper are: 1) an agent-based model, expanded from previous research by some of the authors, which uses values and norms to determine actions on the ultimatum game; 2) a method for estimating an agent’s relative preferences to a given set of values and norms, supported by two approaches to reduce ambiguity in these estimations.

One of the main insights from this work is the need for heterogeneity on observations of human behavior. Whenever these are not present, machines should actively interact with humans. Even in a simple context, ambiguity in these estimations cannot be easily be avoided. To ignore this ambiguity might lead to great regret and misalignment between machine behavior and human values and norms. 

Paper: https://arxiv.org/abs/2004.01056

Presentation: https://underline.io/lecture/97-improving-confidence-in-the-estimation-of-values-and-norms