Project proposal: Corrigibility and interruptibility of homeostasis based agents.
Some of the motivations for solving the problem are:
1. The expected use case properties of the agents: low impact, task-based, soft optimisation / satisficing.
2. Safely getting human feedback to the agent’s behaviour and changing the agent’s goals without the agent trying to manipulate the human’s response too much (reasonable resistance may be permitted).
3. Defining a mitigation against Goodhart’s law. In other words, enabling “common sense” and avoiding a single-dimensional measure of success.